# Data Access Specification

## Purpose

The data access layer provides interface-based repositories with dependency injection for accessing three distinct data sources in the JDE Scoping Tool application:

1. **JDE Oracle** - Primary enterprise system (JD Edwards) for manufacturing data
2. **CMS (Oracle)** - Manufacturing Information System (MIS) data (legacy used DDTek.Oracle driver, migrating to Oracle.ManagedDataAccess.Core)
3. **SQL Server Cache (LotFinderDB)** - Local cache database for search operations

The layer follows a modern repository pattern with:
- **Interface-based repositories** (`ILotFinderRepository`, `IJdeRepository`, `ICmsRepository`) for testability and loose coupling
- **Connection factory abstraction** (`IDbConnectionFactory`) for database connections
- **Dependency injection** via extension methods for service registration
- **Async-first design** with `IAsyncEnumerable<T>` for streaming and `CancellationToken` support
- **Typed exception hierarchy** for consistent error handling

## Source Reference

| Legacy Files | Purpose |
|--------------|---------|
| OLD/DataModel/Process/LotFinderDB.cs | SQL Server cache - base class with connection management |
| OLD/DataModel/Process/LotFinderDB.SearchManagement.cs | SQL Server cache - search CRUD operations |
| OLD/DataModel/Process/LotFinderDB.Item.cs | SQL Server cache - item search/lookup |
| OLD/DataModel/Process/LotFinderDB.WorkOrder.cs | SQL Server cache - work order lookup |
| OLD/DataModel/Process/LotFinderDB.WorkCenter.cs | SQL Server cache - work center search/lookup |
| OLD/DataModel/Process/LotFinderDB.Lot.cs | SQL Server cache - lot lookup |
| OLD/DataModel/Process/LotFinderDB.ProfitCenter.cs | SQL Server cache - profit center search/lookup |
| OLD/DataModel/Process/LotFinderDB.User.cs | SQL Server cache - user search/lookup |
| OLD/DataModel/Process/LotFinderDB.MisData.cs | SQL Server cache - MIS data post-processing |
| OLD/DataModel/Process/JDE.cs | JDE Oracle - base class with connection management |
| OLD/DataModel/Process/JDE.WorkOrders.cs | JDE Oracle - work order queries |
| OLD/DataModel/Process/JDE.WorkOrderStep.cs | JDE Oracle - work order step queries |
| OLD/DataModel/Process/JDE.WorkOrderTime.cs | JDE Oracle - work order time transaction queries |
| OLD/DataModel/Process/JDE.WorkOrderRouting.cs | JDE Oracle - work order routing transaction queries |
| OLD/DataModel/Process/JDE.WorkOrderComponent.cs | JDE Oracle - work order component queries |
| OLD/DataModel/Process/JDE.Lots.cs | JDE Oracle - lot queries |
| OLD/DataModel/Process/JDE.LotUsage.cs | JDE Oracle - lot usage (cardex) queries |
| OLD/DataModel/Process/JDE.LotLocation.cs | JDE Oracle - lot location queries |
| OLD/DataModel/Process/JDE.Items.cs | JDE Oracle - item queries |
| OLD/DataModel/Process/JDE.Users.cs | JDE Oracle - user queries |
| OLD/DataModel/Process/JDE.BusinessUnits.cs | JDE Oracle - branch/profit center/work center queries |
| OLD/DataModel/Process/JDE.StatusCodes.cs | JDE Oracle - status code queries |
| OLD/DataModel/Process/JDE.FunctionCode.cs | JDE Oracle - function code queries |
| OLD/DataModel/Process/JDE.OrgHierarchy.cs | JDE Oracle - organization hierarchy queries |
| OLD/DataModel/Process/JDE.RouteMaster.cs | JDE Oracle - route master queries |
| OLD/DataModel/Process/CMS.cs | CMS - base class with connection management |
| OLD/DataModel/Process/CMS.MisData.cs | CMS - MIS data queries |
| OLD/DataModel/Process/QueryRepository.cs | External SQL query file management |
| OLD/DataModel/Config.cs | Connection string configuration |

---
## Requirements
### Requirement: Repository Interface Pattern

The system SHALL define repository interfaces for all data access operations.

#### Interface Definitions

```csharp
public interface ILotFinderRepository
{
    // Search Management
    Task<List<Search>> GetUserSearchesAsync(string userName, CancellationToken ct = default);
    Task<List<Search>> GetQueuedSearchesAsync(CancellationToken ct = default);
    Task<Search?> GetSearchAsync(int id, CancellationToken ct = default);
    Task<byte[]?> GetSearchResultsAsync(int id, CancellationToken ct = default);
    Task<int> SubmitSearchAsync(Search search, CancellationToken ct = default);
    Task UpdateSearchStatusAsync(int id, SearchStatus status, CancellationToken ct = default);
    Task UpdateSearchResultsAsync(int id, byte[] results, CancellationToken ct = default);

    // Reference Data Lookup
    Task<List<Item>> SearchItemsAsync(string filter, CancellationToken ct = default);
    Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default);
    Task<List<WorkOrder>> LookupWorkordersAsync(List<long> workorderNumbers, CancellationToken ct = default);
    Task<List<WorkCenter>> SearchWorkCentersAsync(string filter, CancellationToken ct = default);
    Task<List<WorkCenter>> LookupWorkCentersAsync(List<string> codes, CancellationToken ct = default);
    Task<List<ProfitCenter>> SearchProfitCentersAsync(string filter, CancellationToken ct = default);
    Task<List<ProfitCenter>> LookupProfitCentersAsync(List<string> codes, CancellationToken ct = default);
    Task<List<JdeUser>> SearchUsersAsync(string filter, CancellationToken ct = default);
    Task<List<JdeUser>> LookupUsersAsync(List<string> userIds, CancellationToken ct = default);
    Task<List<Lot>> LookupLotsAsync(List<LotViewModel> lots, CancellationToken ct = default);

    // Data Sync Operations
    Task<List<DataUpdate>> GetLastDataUpdatesAsync(CancellationToken ct = default);
    Task<TableSpec> GetTableSpecAsync(string tableName, CancellationToken ct = default);
    Task RebuildIndicesAsync(string tableName, CancellationToken ct = default);
    Task PostProcessMisDataAsync(CancellationToken ct = default);
    Task<int> BulkInsertAsync<T>(string tableName, IEnumerable<T> records, CancellationToken ct = default);
    Task TruncateTableAsync(string tableName, CancellationToken ct = default);
}

public interface IJdeRepository
{
    // Work Order Data
    IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrder> GetWorkOrdersArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderRouting> GetWorkOrderRoutingsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);

    // Lot Data
    IAsyncEnumerable<Lot> GetLotsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotUsage> GetLotUsagesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotUsage> GetLotUsagesArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotLocation> GetLotLocationsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);

    // Reference Data
    IAsyncEnumerable<Item> GetItemsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<JdeUser> GetUsersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<Branch> GetBranchesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<ProfitCenter> GetProfitCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkCenter> GetWorkCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<StatusCode> GetStatusCodesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<FunctionCode> GetFunctionCodesAsync(CancellationToken ct = default);
    IAsyncEnumerable<OrgHierarchy> GetOrgHierarchyAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<RouteMaster> GetRouteMastersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
}

public interface ICmsRepository
{
    IAsyncEnumerable<MisData> GetMisDataAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
}
```

#### Business Rules

- All interfaces SHALL be registered with the DI container as scoped services
- Implementations SHALL accept `IDbConnectionFactory` via constructor injection
- Implementations SHALL accept `ILogger<T>` via constructor injection
- Implementations SHALL accept `IOptions<DataAccessOptions>` via constructor injection

#### Scenario: Repository injection
- **WHEN** a service requires data access and requests `ILotFinderRepository` via DI
- **THEN** the container provides a configured `LotFinderRepository` instance with connection factory, logger, and options injected

---

### Requirement: Connection Factory Pattern

The system SHALL provide a connection factory abstraction for database connections.

#### Interface Definition

```csharp
public interface IDbConnectionFactory
{
    Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateJdeConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateJdeStageConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateCmsConnectionAsync(CancellationToken ct = default);
}
```

#### Business Rules

- Connection factory SHALL use `Microsoft.Data.SqlClient.SqlConnection` for SQL Server
- Connection factory SHALL use `Oracle.ManagedDataAccess.Core.OracleConnection` for all Oracle connections (JDE, JDE Stage, CMS)
- Connection strings SHALL be retrieved from `IConfiguration["ConnectionStrings:*"]`
- Secrets (passwords) SHALL be retrieved from Azure Key Vault or .NET Secret Manager (never stored in config files)
- Connections SHALL be opened asynchronously before returning
- Connection factory SHALL be registered as a singleton service

#### Scenario: Create SQL Server connection
- **WHEN** `CreateLotFinderConnectionAsync()` is called
- **THEN** a new `SqlConnection` is created with connection string from configuration, opened asynchronously, and returned

#### Scenario: Create Oracle connection
- **WHEN** `CreateJdeConnectionAsync()` is called
- **THEN** a new `OracleConnection` is created with connection string from configuration, opened asynchronously, and returned

---

### Requirement: Service Registration

The system SHALL provide DI extension methods for service registration.

#### Method Signature

```csharp
public static IServiceCollection AddDataAccess(this IServiceCollection services, IConfiguration configuration)
```

#### Business Rules

- Extension method SHALL register `IDbConnectionFactory` as singleton
- Extension method SHALL register `ILotFinderRepository` as scoped
- Extension method SHALL register `IJdeRepository` as scoped
- Extension method SHALL register `ICmsRepository` as scoped
- Extension method SHALL bind `DataAccessOptions` from configuration section "DataAccess"

#### Scenario: Register data access services
- **WHEN** `services.AddDataAccess(configuration)` is called during startup
- **THEN** all repository interfaces and connection factory are registered with appropriate lifetimes

---

### Requirement: Configuration Options

The system SHALL support configurable timeouts via `IOptions<DataAccessOptions>`.

#### Options Class

```csharp
public class DataAccessOptions
{
    public int DefaultTimeoutSeconds { get; set; } = 600;
    public int LotUsageTimeoutSeconds { get; set; } = 999999;
    public int MisDataTimeoutSeconds { get; set; } = 60000;
    public int RebuildIndexTimeoutSeconds { get; set; } = 600;
    public string ProductionSchema { get; set; } = "PRODDTA";
    public string ArchiveSchema { get; set; } = "ARCDTAPD";
    public string StageSchema { get; set; } = "JDESTAGE";
}
```

#### Business Rules

- Default timeout SHALL be 600 seconds (10 minutes) for general queries
- LotUsage filtered queries SHALL use configurable timeout (default 999999 seconds)
- MIS data queries SHALL use configurable timeout (default 60000 seconds)
- Schema names SHALL be configurable for environment-specific deployments

---

### Requirement: Connection factory pattern

The system SHALL implement `IDbConnectionFactory` to provide database connections via dependency injection.

#### Implementation Pattern

```csharp
public class DbConnectionFactory : IDbConnectionFactory
{
    private readonly IConfiguration _configuration;
    private readonly ILogger<DbConnectionFactory> _logger;

    public DbConnectionFactory(IConfiguration configuration, ILogger<DbConnectionFactory> logger)
    {
        _configuration = configuration;
        _logger = logger;
    }

    public async Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default)
    {
        var connectionString = _configuration.GetConnectionString("LotFinderDB")
            ?? throw new ConnectionException("LotFinderDB connection string not configured", "LotFinderDB");

        var connection = new SqlConnection(connectionString);
        try
        {
            await connection.OpenAsync(ct);
            return connection;
        }
        catch (SqlException ex)
        {
            _logger.LogError(ex, "Failed to connect to LotFinderDB");
            await connection.DisposeAsync();
            throw new ConnectionException("LotFinderDB: failed to open connection to database.", "LotFinderDB", ex);
        }
    }
}
```

#### Business Rules

- Connection factory SHALL be registered as singleton
- Connections SHALL be opened asynchronously before returning
- Callers SHALL dispose returned connections when finished
- `ConnectionException` SHALL be thrown on connection failure with inner exception preserved

#### Scenario: Successful connection creation

- **WHEN** valid connection string exists in configuration
- **AND** `CreateLotFinderConnectionAsync()` is called
- **THEN** a new `SqlConnection` is created, opened, and returned
- **AND** the caller is responsible for disposal

#### Scenario: Connection failure handling

- **WHEN** database is unreachable
- **AND** `CreateLotFinderConnectionAsync()` is called
- **THEN** error is logged with exception details
- **AND** `ConnectionException` is thrown with descriptive message
- **AND** inner exception is preserved for debugging

---

### Requirement: Async streaming pattern

The system SHALL use `IAsyncEnumerable<T>` with Dapper's `QueryUnbufferedAsync` for streaming large result sets.

#### Implementation Pattern

```csharp
public async IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(
    DateTime? lastUpdateDT = null,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateJdeConnectionAsync(ct);

    var sql = ApplySchemaPlaceholders(
        lastUpdateDT.HasValue
            ? JdeQueries.SQL_GET_WORKORDERS_FILTERED
            : JdeQueries.SQL_GET_WORKORDERS);

    var command = new CommandDefinition(
        sql,
        parameters: lastUpdateDT.HasValue
            ? new { dateUpdated = ToJdeDate(lastUpdateDT.Value), timeUpdated = ToJdeTime(lastUpdateDT.Value) }
            : null,
        commandTimeout: _options.Value.DefaultTimeoutSeconds,
        cancellationToken: ct);

    await foreach (var workOrder in connection.QueryUnbufferedAsync<WorkOrder>(command).WithCancellation(ct))
    {
        yield return workOrder;
    }
}
```

#### Business Rules

- All JDE/CMS collection queries SHALL return `IAsyncEnumerable<T>`
- Queries SHALL use `QueryUnbufferedAsync` to stream results
- Methods SHALL accept `CancellationToken` with `[EnumeratorCancellation]` attribute
- Cancellation SHALL be checked between row iterations via `WithCancellation(ct)`

#### Scenario: Stream large work order dataset

- **WHEN** `GetWorkOrdersAsync()` is called for 1 million work orders
- **THEN** results are streamed via `IAsyncEnumerable<WorkOrder>` one at a time
- **AND** memory usage remains constant regardless of result set size
- **AND** consumer can use `await foreach` syntax

#### Scenario: Cancel streaming operation

- **WHEN** cancellation is requested during `GetWorkOrdersAsync()` iteration
- **THEN** iteration stops after current row completes
- **AND** `OperationCanceledException` is thrown to consumer
- **AND** database connection is properly disposed

---

### Requirement: Schema placeholder replacement

The system SHALL replace schema placeholders in SQL queries from `DataAccessOptions`.

#### Implementation Pattern

```csharp
private string ApplySchemaPlaceholders(string sql)
{
    return sql
        .Replace("{ProductionSchema}", _options.Value.ProductionSchema)
        .Replace("{ArchiveSchema}", _options.Value.ArchiveSchema)
        .Replace("{StageSchema}", _options.Value.StageSchema);
}
```

#### Business Rules

- Schema names SHALL be configurable for environment-specific deployments
- Default values: PRODDTA (production), ARCDTAPD (archive), JDESTAGE (stage)
- Replacement SHALL occur at query execution time

#### Scenario: Query uses production schema

- **WHEN** SQL query contains `{ProductionSchema}.F4801`
- **AND** `DataAccessOptions.ProductionSchema` is "PRODDTA"
- **THEN** query is executed with `PRODDTA.F4801`

---

### Requirement: Table name whitelist validation

The system SHALL validate table names against an explicit whitelist to prevent SQL injection in `RebuildIndicesAsync`.

#### Implementation Pattern

```csharp
private static readonly HashSet<string> ValidTableNames = new(StringComparer.OrdinalIgnoreCase)
{
    "Branch", "DataUpdate", "FunctionCode", "Item", "JdeUser",
    "Lot", "LotLocation", "LotUsage_Curr", "LotUsage_Hist",
    "MisData", "OrgHierarchy", "ProfitCenter", "RouteMaster",
    "Search", "StatusCode", "WorkCenter",
    "WorkOrder_Curr", "WorkOrder_Hist",
    "WorkOrderComponent_Curr", "WorkOrderComponent_Hist",
    "WorkOrderRouting",
    "WorkOrderStep_Curr", "WorkOrderStep_Hist",
    "WorkOrderTime_Curr", "WorkOrderTime_Hist"
};

public async Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)
{
    if (!ValidTableNames.Contains(tableName))
    {
        throw new ArgumentException($"Invalid table name: {tableName}", nameof(tableName));
    }

    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
    var sql = $"ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95)";
    await connection.ExecuteAsync(sql, commandTimeout: _options.Value.RebuildIndexTimeoutSeconds);
}
```

#### Business Rules

- Table name MUST be validated against explicit whitelist before execution
- `ArgumentException` SHALL be thrown for invalid table names
- Comparison SHALL be case-insensitive

#### Scenario: Valid table name accepted

- **WHEN** `RebuildIndicesAsync("WorkOrder_Curr")` is called
- **THEN** table name passes whitelist validation
- **AND** index rebuild executes successfully

#### Scenario: SQL injection attempt blocked

- **WHEN** `RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--")` is called
- **THEN** table name fails whitelist validation
- **AND** `ArgumentException` is thrown with message "Invalid table name"
- **AND** no SQL is executed

---

### Requirement: Exception logging with scope context

The system SHALL log exceptions with structured scope context before throwing.

#### Implementation Pattern

```csharp
catch (OracleException ex)
{
    using (_logger.BeginScope(new Dictionary<string, object>
    {
        ["DataSource"] = "JDE",
        ["Operation"] = nameof(GetWorkOrdersAsync),
        ["QueryName"] = "SQL_GET_WORKORDERS"
    }))
    {
        _logger.LogError(ex, "Query execution failed");
    }
    throw new QueryException("Failed to execute work order query", "SQL_GET_WORKORDERS", ex);
}
```

#### Business Rules

- All exceptions SHALL be logged at throw site
- Log context SHALL include: DataSource, Operation, and QueryName where applicable
- Inner exceptions SHALL be preserved in thrown exceptions
- Structured logging SHALL enable log aggregation and analysis

#### Scenario: Query exception with full context

- **WHEN** JDE Oracle query fails with OracleException
- **THEN** error is logged with BeginScope containing DataSource, Operation, QueryName
- **AND** `QueryException` is thrown with descriptive message and inner exception

---

### Requirement: Search criteria extraction via SQL functions

The system SHALL use SQL extraction functions to retrieve filter criteria directly from the `Search.Criteria` JSON column.

#### Implementation Pattern

Search query building now uses SearchId to invoke extraction functions rather than passing filter values from C#:

```csharp
public SearchQueryResult BuildSearchQuery(int searchId)
{
    // Query builder generates SQL that calls extraction functions
    // Example generated SQL fragment:
    // INSERT INTO #P_WorkOrders SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(@SearchId)
    // INSERT INTO #P_ItemNumbers SELECT ItemNumber FROM dbo.fn_GetSearchItemNumbers(@SearchId)

    return new SearchQueryResult(sql, new { SearchId = searchId });
}
```

#### Business Rules

- Query builder SHALL accept only `searchId` parameter (not full criteria object)
- SQL queries SHALL call extraction functions to populate temporary filter tables
- Extraction functions handle JSON parsing and validation in SQL Server
- Invalid JSON or missing criteria results in empty filter sets (no errors thrown)

#### Available Extraction Functions

| Function | Returns |
|----------|---------|
| `fn_GetSearchMinimumDt` | DATETIME2 scalar |
| `fn_GetSearchMaximumDt` | DATETIME2 scalar |
| `fn_GetSearchExtractMisData` | BIT scalar |
| `fn_GetSearchWorkOrders` | WorkOrderNumber table |
| `fn_GetSearchItemNumbers` | ItemNumber table |
| `fn_GetSearchProfitCenters` | Code table |
| `fn_GetSearchWorkCenters` | Code table |
| `fn_GetSearchOperatorIDs` | OperatorID table |
| `fn_GetSearchComponentLots` | LotNumber, ItemNumber table |
| `fn_GetSearchPartOperations` | ItemNumber, OperationNumber, MisNumber, MisRevision table |

#### Scenario: Build search query with extraction functions

- **WHEN** `BuildSearchQuery(123)` is called for a search with work order filter
- **THEN** generated SQL includes `SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(123)`
- **AND** only the `@SearchId` parameter is passed to the query

---

### Requirement: Table-valued parameter support for lookups

The system SHALL use DataTable for SQL Server table-valued parameters in reference data lookup methods.

#### Business Rules

- TVPs are used for batch lookups (LookupItemsAsync, LookupWorkordersAsync, etc.)
- TVPs are NOT used for search query execution (replaced by extraction functions)
- DataTable column names SHALL match TVP type column names

#### Scenario: Batch lookup with TVP

- **WHEN** `LookupItemsAsync(["ITEM001", "ITEM002", "ITEM003"])` is called
- **THEN** DataTable is created with ItemNumber column
- **AND** single query executes with TVP parameter
- **AND** matching items are returned

---

### Requirement: Service registration extension method

The system SHALL provide `AddDataAccess` extension method for DI registration.

#### Implementation Pattern

```csharp
public static IServiceCollection AddDataAccess(
    this IServiceCollection services,
    IConfiguration configuration)
{
    services.Configure<DataAccessOptions>(
        configuration.GetSection(DataAccessOptions.SectionName));

    services.AddSingleton<IDbConnectionFactory, DbConnectionFactory>();
    services.AddScoped<ILotFinderRepository, LotFinderRepository>();
    services.AddScoped<IJdeRepository, JdeRepository>();
    services.AddScoped<ICmsRepository, CmsRepository>();

    return services;
}
```

#### Business Rules

- Extension method SHALL bind `DataAccessOptions` from "DataAccess" configuration section
- Connection factory SHALL be registered as singleton
- Repositories SHALL be registered as scoped services
- Method SHALL return `IServiceCollection` for chaining

#### Scenario: Register all data access services

- **WHEN** `services.AddDataAccess(configuration)` is called during startup
- **THEN** `DataAccessOptions` is bound from configuration
- **AND** `IDbConnectionFactory` is registered as singleton
- **AND** all repository interfaces are registered as scoped

## Exception Handling

### Requirement: Custom Exception Types

The system SHALL define custom exception types for data access errors.

#### Exception Hierarchy

```csharp
public class DataAccessException : Exception
{
    public string? Operation { get; }
    public string? Repository { get; }
    public DataAccessException(string message, string? operation = null, string? repository = null, Exception? inner = null);
}

public class ConnectionException : DataAccessException
{
    public string? DataSource { get; }
    public ConnectionException(string message, string dataSource, Exception? inner = null);
}

public class QueryException : DataAccessException
{
    public string? QueryName { get; }
    public QueryException(string message, string queryName, Exception? inner = null);
}

public class DataAccessTimeoutException : DataAccessException
{
    public int TimeoutSeconds { get; }
    public DataAccessTimeoutException(string message, int timeoutSeconds, Exception? inner = null);
}
```

#### Business Rules

- All repository methods SHALL throw typed exceptions on error (never return null/empty on error)
- `ConnectionException` SHALL be thrown for connection failures
- `QueryException` SHALL be thrown for query execution failures
- `DataAccessTimeoutException` SHALL be thrown for timeout errors
- Exception SHALL be logged at throw site via `ILogger<T>` with `BeginScope()` for context
- Inner exceptions SHALL be preserved for debugging

#### Scenario: Connection failure logging
- **WHEN** a connection to JDE Oracle fails
- **THEN** error is logged with scope context (data source, operation) and `ConnectionException` is thrown with descriptive message and inner exception

#### Scenario: Query timeout
- **WHEN** a query exceeds configured timeout
- **THEN** `DataAccessTimeoutException` is thrown with timeout value and query name

---

## Query Management

### Requirement: QueryRepository component

The system SHALL manage SQL queries as embedded resources or compile-time constants.

#### Business Rules

- SQL queries SHALL be stored as embedded resources in the assembly or as `const string` fields
- Queries SHALL be loaded once at application startup and cached in memory
- Schema placeholder replacement (`PRODDTA`, `ARCDTAPD`, `JDESTAGE`) SHALL use values from `DataAccessOptions`
- Query loading SHALL throw `QueryException` if query is not found

#### Scenario: Load query from embedded resource
- **WHEN** repository is instantiated and SQL queries are needed
- **THEN** queries are loaded from embedded resources with schema placeholders replaced from configuration

#### Scenario: Query not found
- **WHEN** a requested query name does not exist in embedded resources
- **THEN** `QueryException` is thrown with message containing the query name

---

## Connection Management

### Requirement: Connection Management - LotFinderDB component

The system SHALL provide SQL Server cache database connections via `IDbConnectionFactory`.

#### Method: CreateLotFinderConnectionAsync

Opens a new connection to the LotFinderDB SQL Server database.

| Parameter | Type | Description |
|-----------|------|-------------|
| ct | CancellationToken | Cancellation token for async operation |

| Returns | Type | Description |
|---------|------|-------------|
| connection | Task<SqlConnection> | Open connection to LotFinderDB |

#### Business Rules

- Connection string loaded from `IConfiguration["ConnectionStrings:LotFinderDB"]`
- Password retrieved from Azure Key Vault or .NET Secret Manager
- Uses `Microsoft.Data.SqlClient.SqlConnection`
- Returns opened connection (caller must dispose)
- Throws `ConnectionException` on failure with descriptive message
- Default command timeout: configurable via `DataAccessOptions.DefaultTimeoutSeconds`

#### Scenario: Successful connection
- **WHEN** valid LotFinderDB connection string exists in configuration and `CreateLotFinderConnectionAsync()` is called
- **THEN** a new `SqlConnection` is created, opened asynchronously, and returned

#### Scenario: Connection failure
- **WHEN** invalid or unreachable database exists and `CreateLotFinderConnectionAsync()` is called
- **THEN** error is logged via `ILogger<T>` and `ConnectionException` is thrown with message "LotFinderDB: failed to open connection to database."

---

### Requirement: Connection Management - JDE component

The system SHALL provide JDE Oracle database connections via `IDbConnectionFactory`.

#### Method: CreateJdeConnectionAsync

Opens a new connection to the JDE Oracle database.

| Parameter | Type | Description |
|-----------|------|-------------|
| ct | CancellationToken | Cancellation token for async operation |

| Returns | Type | Description |
|---------|------|-------------|
| connection | Task<OracleConnection> | Open connection to JDE (Oracle.ManagedDataAccess.Core) |

#### Business Rules

- Connection string loaded from `IConfiguration["ConnectionStrings:JDE"]`
- Password retrieved from Azure Key Vault or .NET Secret Manager
- Uses `Oracle.ManagedDataAccess.Core.OracleConnection`
- Returns opened connection (caller must dispose)
- Throws `ConnectionException` on failure

#### Scenario: Successful JDE connection
- **WHEN** valid JDE Oracle connection string exists and `CreateJdeConnectionAsync()` is called
- **THEN** a new OracleConnection is created, opened asynchronously, and returned

---

### Requirement: Connection Management - JDE Stage component

The system SHALL provide a separate JDE connection for stage/view queries.

#### Method: CreateJdeStageConnectionAsync

Opens a new connection to the JDE Stage database (for JDESTAGE schema views).

#### Business Rules

- Connection string loaded from `IConfiguration["ConnectionStrings:JDEStage"]`
- Uses `Oracle.ManagedDataAccess.Core.OracleConnection`
- Used for `GetStatusCodes()` and `GetLotLocations()` methods
- Queries JDESTAGE schema views (F0005_VIEW, F41021_VIEW)

---

### Requirement: Connection Management - CMS component

The system SHALL provide CMS database connections via `IDbConnectionFactory`.

#### Method: CreateCmsConnectionAsync

Opens a new connection to the CMS database.

| Parameter | Type | Description |
|-----------|------|-------------|
| ct | CancellationToken | Cancellation token for async operation |

| Returns | Type | Description |
|---------|------|-------------|
| connection | Task<OracleConnection> | Open connection to CMS |

#### Business Rules

- Connection string loaded from `IConfiguration["ConnectionStrings:CMS"]`
- Password retrieved from Azure Key Vault or .NET Secret Manager
- Uses `Oracle.ManagedDataAccess.Core.OracleConnection` (consolidated from legacy DDTek.Oracle)

---

## Async Streaming

### Requirement: IAsyncEnumerable Streaming

The system SHALL use `IAsyncEnumerable<T>` for streaming large datasets.

#### Business Rules

- All JDE/CMS query methods returning collections SHALL return `IAsyncEnumerable<T>`
- Implementations SHALL use Dapper's `QueryUnbufferedAsync` for streaming results
- All streaming methods SHALL accept `CancellationToken` parameter
- Cancellation SHALL be checked between row iterations

#### Scenario: Stream work orders
- **WHEN** `GetWorkOrdersAsync()` is called with large dataset
- **THEN** results are streamed via `IAsyncEnumerable<WorkOrder>` without loading all rows into memory

#### Scenario: Cancel streaming operation
- **WHEN** cancellation is requested during `GetWorkOrdersAsync()` iteration
- **THEN** iteration stops and `OperationCanceledException` is thrown

---

## JDE Oracle Queries

### Requirement: JDE.GetWorkOrders component

The system SHALL fetch work order records from JDE F4801 table.

#### Method Signature

```csharp
IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| lastUpdateDT | DateTime? | Optional cutoff for incremental sync |
| ct | CancellationToken | Cancellation token |

#### Returns

- `IAsyncEnumerable<WorkOrder>` - Streaming results via QueryUnbufferedAsync

#### Query: SQL_GET_WORKORDERS

```sql
SELECT wo.WADOCO AS WorkOrderNumber,
       TRIM(wo.WAMMCU) AS BranchCode,
       TRIM(wo.WALOTN) AS LotNumber,
       TRIM(wo.WALITM) AS ItemNumber,
       wo.WAITM AS ShortItemNumber,
       TRIM(wo.WAPARS) AS ParentWorkOrderNumber,
       wo.WAUORG / 100.0 AS OrderQuantity,
       wo.WASOBK / 100.0 AS HeldQuantity,
       wo.WASOQS / 100.0 AS ShippedQuantity,
       TRIM(wo.WASRST) AS StatusCode,
       CASE wo.WADCG WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WADCG+1900000,'YYYYDDD') END AS StatusCodeUpdateDT,
       CASE wo.WATRDJ WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WATRDJ+1900000,'YYYYDDD') END AS IssueDate,
       CASE wo.WASTRT WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WASTRT+1900000,'YYYYDDD') END AS StartDate,
       TRIM(wo.WATRT) AS RoutingType,
       wo.WAUPMJ AS LastUpdateDate,
       wo.WATDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4801 wo
```

#### Query: SQL_GET_WORKORDERS_FILTERED (Incremental)

Same SELECT with WHERE clause:
```sql
WHERE (wo.WAUPMJ > :dateUpdated OR
       (wo.WAUPMJ = :dateUpdated AND wo.WATDAY >= :timeUpdated))
```

#### Query: SQL_GET_WORKORDERS_ARCHIVE

Same SELECT from `{ArchiveSchema}.F4801` (archive schema)

#### JDE Table Mapping

| JDE Column | Domain Property | Transformation |
|------------|-----------------|----------------|
| WADOCO | WorkOrderNumber | Direct |
| WAMMCU | BranchCode | TRIM |
| WALOTN | LotNumber | TRIM |
| WALITM | ItemNumber | TRIM |
| WAITM | ShortItemNumber | Direct |
| WAPARS | ParentWorkOrderNumber | TRIM |
| WAUORG | OrderQuantity | / 100.0 |
| WASOBK | HeldQuantity | / 100.0 |
| WASOQS | ShippedQuantity | / 100.0 |
| WASRST | StatusCode | TRIM |
| WADCG | StatusCodeUpdateDT | JDE date conversion |
| WATRDJ | IssueDate | JDE date conversion |
| WASTRT | StartDate | JDE date conversion |
| WATRT | RoutingType | TRIM |
| WAUPMJ | LastUpdateDate | JDE date (integer) |
| WATDAY | LastUpdateTime | JDE time (integer) |

#### Business Rules

- Quantities stored in JDE as integer * 100, divided for decimal values
- JDE dates stored as integer CYYDDD format, converted to DATE
- Zero dates converted to '1900-01-01'
- Uses configurable query timeout from `DataAccessOptions.DefaultTimeoutSeconds`
- Streaming results via `QueryUnbufferedAsync` for memory efficiency
- Schema placeholder `{ProductionSchema}` replaced from configuration

#### Scenario: Full sync (mass update)
- **WHEN** lastUpdateDT is null and `GetWorkOrdersAsync()` is called
- **THEN** SQL_GET_WORKORDERS query returns all work orders from JDE as async stream

#### Scenario: Incremental sync (daily/hourly update)
- **WHEN** lastUpdateDT is 2024-01-15 and `GetWorkOrdersAsync(lastUpdateDT)` is called
- **THEN** SQL_GET_WORKORDERS_FILTERED returns only records updated since that date

#### Scenario: JDE quantity conversion
- **WHEN** JDE WAUORG value is 10050 (integer) and mapping to WorkOrder.OrderQuantity
- **THEN** OrderQuantity equals 100.50 (decimal, divided by 100)

---

### Requirement: JDE.GetWorkOrderSteps component

The system SHALL fetch work order operation steps from JDE F3112 table.

#### Method Signature

```csharp
IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_WORKORDER_STEP

```sql
SELECT wos.WLDOCO AS WorkOrderNumber,
       wos.WLOPSQ/10 AS StepNumber,
       TRIM(wos.WLMCU) AS WorkCenterCode,
       TRIM(wos.WLMMCU) AS BranchCode,
       TRIM(wos.WLDSC1) AS StepDescription,
       TRIM(mes.CFDS80) AS FunctionOperationDescription,
       wos.WLOPSC AS StepTypeCode,
       CASE wos.WLSTRT WHEN 0 THEN NULL
            ELSE TO_DATE(wos.WLSTRT+1900000,'YYYYDDD') END AS StartDT,
       CASE wos.WLSTRX WHEN 0 THEN NULL
            ELSE TO_DATE(wos.WLSTRX+1900000,'YYYYDDD') END AS EndDT,
       TRIM(wos.WLURRF) AS FunctionCode,
       wos.WLSOCN / 100.0 AS ScrappedQuantity,
       wos.WLUPMJ AS LastUpdateDate,
       wos.WLTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3112 wos LEFT OUTER JOIN
       {ProductionSchema}.F00192 mes ON (wos.WLURRF = mes.CFKY)
 WHERE TRIM(wos.WLMCU) IS NOT NULL AND
       TRIM(wos.WLMMCU) IS NOT NULL
```

#### Business Rules

- StepNumber is WLOPSQ/10 (stored as tenths in JDE)
- LEFT OUTER JOIN to F00192 for FunctionOperationDescription
- Filters out records with NULL work center or branch
- Archive queries use `{ArchiveSchema}.F3112`

#### Scenario: Step number conversion
- **WHEN** JDE WLOPSQ value is 100 (tenths) and mapping to WorkOrderStep.StepNumber
- **THEN** StepNumber equals 10.0 (decimal)

#### Scenario: Enrich with function description
- **WHEN** step has FunctionCode "ASSY" and LEFT OUTER JOIN to F00192 succeeds
- **THEN** FunctionOperationDescription is populated from media object

---

### Requirement: JDE.GetWorkOrderTimes component

The system SHALL fetch work order time transactions (operator labor) from JDE F31122 table.

#### Method Signature

```csharp
IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_WORKORDER_TIMES

```sql
SELECT wot.WTUKID AS UniqueID,
       wot.WTDOCO AS WorkOrderNumber,
       wot.WTOPSQ/10 AS StepNumber,
       TRIM(wot.WTMCU) AS WorkCenterCode,
       TRIM(wot.WTMMCU) AS BranchCode,
       wot.WTAN8 AS AddressNumber,
       CASE wot.WTDGL WHEN 0 THEN NULL
            ELSE TO_DATE(wot.WTDGL+1900000,'YYYYDDD') END AS GlDate,
       wot.WTUPMJ AS LastUpdateDate,
       wot.WTTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F31122 wot
 WHERE TRIM(wot.WTMCU) IS NOT NULL AND
       TRIM(wot.WTMMCU) IS NOT NULL
```

#### Business Rules

- Links operators to work orders via AddressNumber -> JdeUser
- Filters out records with NULL work center or branch

---

### Requirement: JDE.GetWorkOrderRoutings component

The system SHALL fetch work order routing transactions from JDE F3112Z1 table.

#### Method Signature

```csharp
IAsyncEnumerable<WorkOrderRouting> GetWorkOrderRoutingsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_WORKORDER_ROUTING

```sql
SELECT TRIM(woz.SZEDUS) AS UserID,
       TRIM(woz.SZEDBT) AS BatchNumber,
       TRIM(woz.SZEDTN) AS TransactionNumber,
       woz.SZEDLN AS LineNumber,
       woz.SZOPSQ / 10.0 AS StepNumber,
       TRIM(woz.SZMCU) AS WorkCenterCode,
       woz.SZDOCO AS WorkOrderNumber,
       TRIM(woz.SZTRT) AS RoutingType,
       TRIM(woz.SZMMCU) AS BranchCode,
       TRIM(woz.SZDSC1) AS StepDescription,
       TRIM(woz.SZURRF) AS FunctionCode,
       woz.SZTRDJ AS TransactionDate_Date,
       woz.SZUPMJ AS LastUpdateDate,
       woz.SZTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3112Z1 woz
 WHERE woz.SZTYTN = 'JDERTG' AND
       woz.SZDRIN = '2' AND
       woz.SZTNAC = '02' AND
       woz.SZPID = 'ER31410' AND
       TRIM(woz.SZEDUS) IS NOT NULL AND
       TRIM(woz.SZEDBT) IS NOT NULL AND
       TRIM(woz.SZEDTN) IS NOT NULL AND
       TRIM(woz.SZMCU) IS NOT NULL
```

#### Business Rules

- Filters by specific transaction type (JDERTG), direction (2), action (02), program ID (ER31410)
- Used for MIS matching to find original operation sequence
- **Data validation**: Skips records where LastUpdateDT or TransactionDate year < 1900 or > 2500
- No archive table (single table for all data)

---

### Requirement: JDE.GetWorkOrderComponents component

The system SHALL fetch work order component usage from JDE F3111 table.

#### Method Signature

```csharp
IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_WORKORDER_COMPONENTS

```sql
SELECT woc.WMUKID AS UniqueID,
       woc.WMDOCO AS WorkOrderNumber,
       TRIM(woc.WMLOTN) AS LotNumber,
       TRIM(woc.WMCMCU) AS BranchCode,
       woc.WMCPIT AS ShortItemNumber,
       woc.WMTRQT / 100.0 AS Quantity,
       woc.WMUPMJ AS LastUpdateDate,
       woc.WMTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3111 woc
 WHERE TRIM(woc.WMLOTN) IS NOT NULL
```

#### Business Rules

- Filters records with NULL lot number
- Archive queries use `{ArchiveSchema}.F3111`

---

### Requirement: JDE.GetLots component

The system SHALL fetch lot master data from JDE F4108 table.

#### Method Signature

```csharp
IAsyncEnumerable<Lot> GetLotsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_LOTS

```sql
SELECT TRIM(lot.IOLOTN) AS LotNumber,
       TRIM(lot.IOMCU) AS BranchCode,
       lot.IOITM AS ShortItemNumber,
       TRIM(lot.IOLITM) AS ItemNumber,
       lot.IOVEND AS SupplierCode,
       lot.IOLOTS AS StatusCode,
       lot.IOLOT1 AS Memo1,
       lot.IOLOT2 AS Memo2,
       lot.IOLOT3 AS Memo3,
       lot.IOUPMJ AS LastUpdateDate,
       lot.IOTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4108 lot
 WHERE TRIM(lot.IOLOTN) IS NOT NULL AND
       TRIM(lot.IOMCU) IS NOT NULL
```

#### Business Rules

- StatusCode (IOLOTS) is single character
- Filters out records with NULL lot number or branch code

---

### Requirement: JDE.GetLotUsages component

The system SHALL fetch lot usage (cardex) transactions from JDE F4111 table.

#### Method Signature

```csharp
IAsyncEnumerable<LotUsage> GetLotUsagesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_LOT_USAGES

```sql
SELECT lu.ILUKID AS UniqueID,
       lu.ILDOCO AS WorkOrderNumber,
       TRIM(lu.ILLOTN) AS LotNumber,
       TRIM(lu.ILMCU) AS BranchCode,
       lu.ILITM AS ShortItemNumber,
       lu.ILTRQT AS Quantity,
       lu.ILTRDJ AS LastUpdateDate,
       lu.ILTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4111 lu
 WHERE lu.ILDCT = 'IM' AND
       TRIM(lu.ILLOTN) IS NOT NULL
```

#### Business Rules

- Filters by document type 'IM' (inventory management)
- Filters out records with NULL lot number
- **Special timeout**: Filtered query uses `DataAccessOptions.LotUsageTimeoutSeconds` (default 999999)
- Archive queries use `{ArchiveSchema}.F4111`

---

### Requirement: JDE.GetLotLocations component

The system SHALL fetch lot location tracking from JDESTAGE.F41021_VIEW.

#### Method Signature

```csharp
IAsyncEnumerable<LotLocation> GetLotLocationsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_LOT_LOCATIONS

```sql
SELECT TRIM(il.LOT_LILOTN) AS LotNumber,
       il.IDENTIFIERSHORTITEM_LIITM AS ShortItemNumber,
       TRIM(il.COSTCENTER_LIMCU) AS BranchCode,
       COALESCE(TRIM(il.LOCATION_LILOCN), ' ') AS Location,
       il.DATEUPDATED_LIUPMJ + FLOOR(il.TIMEOFDAY_LITDAY / 10000) / 24 +
         FLOOR(MOD(il.TIMEOFDAY_LITDAY, 10000) / 100) / 1440 +
         MOD(il.TIMEOFDAY_LITDAY, 100) / 86400 AS LastUpdateDT
  FROM {StageSchema}.F41021_VIEW il
 WHERE TRIM(il.LOT_LILOTN) IS NOT NULL
```

#### Business Rules

- Uses JDESTAGE view via `CreateJdeStageConnectionAsync()` (different connection than main JDE)
- Location defaults to space if NULL
- LastUpdateDT computed inline from JDE date/time integers

---

### Requirement: JDE.GetItems component

The system SHALL fetch item (part number) master data from JDE F4101 table.

#### Method Signature

```csharp
IAsyncEnumerable<Item> GetItemsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_ITEMS

```sql
SELECT pn.IMITM AS ShortItemNumber,
       TRIM(pn.IMLITM) AS ItemNumber,
       TRIM(pn.IMDSC1) AS Description,
       TRIM(pn.IMPRP4) AS PlanningFamily,
       TRIM(pn.IMSTKT) AS StockingType,
       pn.IMUPMJ AS LastUpdateDate,
       pn.IMTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4101 pn
 WHERE TRIM(pn.IMLITM) IS NOT NULL
```

---

### Requirement: JDE.GetUsers component

The system SHALL fetch user/operator data from JDE F0101 and SY920.F0092 tables.

#### Method Signature

```csharp
IAsyncEnumerable<JdeUser> GetUsersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_USERS

```sql
WITH USER_CTE AS (
  SELECT ab.ABAN8 AS AddressNumber,
         TRIM(pro.ULUSER) AS UserID,
         TRIM(ab.ABALPH) AS FullName,
         ab.ABUPMJ AS LastUpdateDate,
         ab.ABUPMT AS LastUpdateTime,
         ROW_NUMBER() OVER (PARTITION BY ab.ABAN8 ORDER BY ab.ABUPMJ DESC, ab.ABUPMT DESC) RN
   FROM  {ProductionSchema}.F0101 ab LEFT OUTER JOIN
         SY920.F0092 pro ON (ab.ABAN8 = pro.ULAN8)
)
SELECT AddressNumber,
       UserID,
       FullName,
       LastUpdateDate,
       LastUpdateTime
  FROM USER_CTE
 WHERE RN = 1
```

#### Business Rules

- CTE with ROW_NUMBER to get latest record per AddressNumber
- LEFT OUTER JOIN to SY920 schema for UserID lookup
- **Note**: Incremental filtering not supported for users (full sync always)

---

### Requirement: JDE.GetBranches / GetProfitCenters / GetWorkCenters component

The system SHALL fetch business unit reference data from JDE F0006 table.

#### Method Signature

```csharp
IAsyncEnumerable<Branch> GetBranchesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
IAsyncEnumerable<ProfitCenter> GetProfitCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
IAsyncEnumerable<WorkCenter> GetWorkCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_BUSINESS_UNITS

```sql
SELECT TRIM(wc.MCMCU) AS Code,
       TRIM(wc.MCDL01) AS Description,
       wc.MCUPMJ AS LastUpdateDate,
       wc.MCUPMT AS LastUpdateTime
  FROM {ProductionSchema}.F0006 wc
 WHERE wc.MCSTYL = :typeCode
```

#### Parameters

| Entity | typeCode Parameter |
|--------|-------------------|
| Branch | 'BP' |
| ProfitCenter | 'I3' |
| WorkCenter | 'WC' |

---

### Requirement: JDE.GetStatusCodes component

The system SHALL fetch work order status codes from JDESTAGE.F0005_VIEW.

#### Method Signature

```csharp
IAsyncEnumerable<StatusCode> GetStatusCodesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_STATUS_CODES

```sql
SELECT TRIM(sc.USERDEFINEDCODE_DRKY) AS CODE,
       TRIM(sc.DESCRIPTION001_DRDL01) AS Description,
       sc.DATEUPDATED_DRUPMJ + FLOOR(sc.TIMELASTUPDATED_DRUPMT / 10000) / 24 +
         FLOOR(MOD(sc.TIMELASTUPDATED_DRUPMT, 10000) / 100) / 1440 +
         MOD(sc.TIMELASTUPDATED_DRUPMT, 100) / 86400 AS LastUpdateDT
  FROM {StageSchema}.F0005_VIEW sc
 WHERE TRIM(sc.PRODUCTCODE_DRSY) = '00' AND
       sc.USERDEFINEDCODES_DRRT = 'SS' AND
       TRIM(sc.USERDEFINEDCODE_DRKY) IS NOT NULL
```

#### Business Rules

- Uses JDE Stage connection via `CreateJdeStageConnectionAsync()` (different connection than main JDE)
- Filters by product code '00' and UDC type 'SS'
- LastUpdateDT computed inline from date/time integers

---

### Requirement: JDE.GetFunctionCodes component

The system SHALL fetch function code lookup data from JDE F00192 table.

#### Method Signature

```csharp
IAsyncEnumerable<FunctionCode> GetFunctionCodesAsync(CancellationToken ct = default)
```

#### Query: SQL_GET_FUNCTION_CODES

```sql
SELECT Code,
       TRIM(LISTAGG(Description, ' ') WITHIN GROUP(ORDER BY Description) ||
         CASE WHEN MAX(total_lengthb) > 4000 THEN '...' ELSE '' END) Description,
       SYSDATE AS LastUpdateDT
FROM (
  SELECT TRIM(fc.CFKY) AS Code,
         TRIM(ASCIISTR(fc.CFDS80)) AS Description,
         SUM(LENGTHB(TRIM(fc.CFDS80))+1) OVER(PARTITION BY TRIM(fc.CFKY)
           ORDER BY TRIM(fc.CFDS80)) - 1 cumul_lengthb,
         SUM(LENGTHB(TRIM(fc.CFDS80))+1) OVER(PARTITION BY TRIM(fc.CFKY)) - 1 total_lengthb,
         COUNT(*) OVER(PARTITION BY TRIM(fc.CFKY)) num_values
  FROM   {ProductionSchema}.F00192 fc
  WHERE  TRIM(fc.CFKY) IS NOT NULL
)
WHERE total_lengthb <= 4000 OR cumul_lengthb <= 4000 - length('...')
GROUP BY Code
```

#### Business Rules

- Aggregates multiple description rows per code using LISTAGG
- Truncates at 4000 bytes with '...' suffix
- Uses ASCIISTR to handle special characters
- LastUpdateDT set to SYSDATE (current timestamp)
- **Note**: Does not support incremental filtering (no lastUpdateDT parameter)

---

### Requirement: JDE.GetOrgHierarchy component

The system SHALL fetch organization hierarchy (work center to profit center mapping) from JDE F30006 table.

#### Method Signature

```csharp
IAsyncEnumerable<OrgHierarchy> GetOrgHierarchyAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_ORG_HIERARCHY

```sql
SELECT TRIM(oh.IWMCUW) AS ProfitCenterCode,
       TRIM(oh.IWMCU) AS WorkCenterCode,
       TRIM(oh.IWMMCU) AS BranchCode,
       oh.IWUPMJ AS LastUpdateDate,
       oh.IWTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F30006 oh
 WHERE TRIM(oh.IWMCU) IS NOT NULL AND
       TRIM(oh.IWMMCU) IS NOT NULL
```

---

### Requirement: JDE.GetRouteMasters component

The system SHALL fetch item routing master data from JDE F3003 table.

#### Method Signature

```csharp
IAsyncEnumerable<RouteMaster> GetRouteMastersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_ROUTE_MASTER

```sql
SELECT TRIM(rm.IRMMCU) AS BranchCode,
       TRIM(rm.IRKITL) AS ItemNumber,
       TRIM(rm.IRTRT) AS RoutingType,
       rm.IROPSQ / 10.0 AS SequenceNumber,
       TRIM(rm.IRURRF) AS FunctionCode,
       TRIM(rm.IRMCU) AS WorkCenterCode,
       rm.IREFFF AS StartDate_Date,
       rm.IREFFT AS EndDate_Date,
       rm.IRUPMJ AS LastUpdateDate,
       rm.IRTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3003 rm
 WHERE TRIM(rm.IRKITL) IS NOT NULL
```

#### Business Rules

- SequenceNumber is IROPSQ/10 (stored as tenths)
- StartDate_Date and EndDate_Date are JDE integer format (converted in model)

---

## CMS Queries

### Requirement: CMS.GetMisData component

The system SHALL fetch Manufacturing Information System (MIS) data from CMS database.

#### Method Signature

```csharp
IAsyncEnumerable<MisData> GetMisDataAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
```

#### Query: SQL_GET_MIS_DATA

```sql
SELECT DISTINCT
       mis.P_PART_NUMBER AS ItemNumber,
       mis.P_OPERATION_NUMBER AS SequenceNumber,
       item.PITEM_ID AS MISNumber,
       itemrev.PITEM_REVISION_ID AS RevID,
       TRIM(mis.P_SITE) AS BranchCode,
       zim_test_details.P_SEQ_NUMBER AS CharNumber,
       zim_test_details.P_TEST_DESC AS TestDescription,
       zim_test_details.P_SAMPL_TYPE AS SamplingType,
       zim_test_details.P_SAMPL_VALUE AS SamplingValue,
       zim_test_details.P_TOOLS AS ToolsGauges,
       zim_test_details.P_WORK_INTR AS WorkInstructions,
       Status.PNAME AS Status,
       Status.PDATE_RELEASED AS ReleaseDate
  FROM INFODBA.PITEM item
       INNER JOIN INFODBA.PITEMREVISION itemrev ON (item.PUID = itemrev.RITEMS_TAGU)
       INNER JOIN INFODBA.PRELEASE_STATUS_LIST listing ON (itemrev.PUID = listing.PUID)
       INNER JOIN INFODBA.PRELEASESTATUS Status ON (listing.PVALU_0 = Status.PUID)
       INNER JOIN INFODBA.PIMANRELATION imanrel ON (itemrev.PUID = imanrel.RPRIMARY_OBJECTU)
       INNER JOIN INFODBA.PFORM form ON (imanrel.RSECONDARY_OBJECTU = form.PUID)
       INNER JOIN INFODBA.PZIMMERMISDETAILS zim_mis ON (form.RDATA_FILEU = zim_mis.PUID)
       INNER JOIN INFODBA.P_TEST_DETAILS test_details ON (zim_mis.PUID = test_details.PUID)
       INNER JOIN INFODBA.P_PART_ASSOCIATION ppa ON (ppa.PUID = test_details.PUID)
       INNER JOIN INFODBA.PMISDATAOBJECT mis ON (mis.PUID = ppa.PVALU_0)
       INNER JOIN INFODBA.PZIMTESTDETAILS zim_test_details ON (test_details.PVALU_0 = zim_test_details.PUID)
 WHERE Status.PNAME IN ('Current', 'BackLevel')
```

#### Business Rules

- Complex 10-table JOIN through CMS schema (INFODBA)
- Filters for Status 'Current' or 'BackLevel' only
- **Special timeout**: Uses `DataAccessOptions.MisDataTimeoutSeconds` (default 60000 seconds)
- **Data transformation**: ReleaseDate converted to local time
- Uses CMS connection via `CreateCmsConnectionAsync()`

#### Scenario: Fetch MIS data from CMS
- **WHEN** CMS connection is available and `GetMisDataAsync()` is called
- **THEN** 10-table JOIN returns Current and BackLevel MIS records as async stream

#### Scenario: Handle long-running query
- **WHEN** MIS data query may take extended time and query executes
- **THEN** configured timeout allows completion without error

---

## SQL Server Cache Operations

### Requirement: LotFinderDB.GetUserSearches component

The system SHALL retrieve searches for a specific user.

#### Method Signature

```csharp
Task<List<Search>> GetUserSearchesAsync(string userName, CancellationToken ct = default)
```

#### Query

```sql
SELECT s.ID, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT
FROM   dbo.Search s
WHERE  s.UserName = @userName
ORDER BY s.SubmitDT
```

#### Business Rules

- Throws `QueryException` on error (does not return empty list on error)
- Does not include Criteria or Results (lightweight query)
- Ordered by SubmitDT ascending

#### Scenario: Get user's search history
- **WHEN** user "jdoe" has 5 previous searches and `GetUserSearchesAsync("jdoe")` is called
- **THEN** 5 Search objects are returned without Criteria/Results (lightweight)

---

### Requirement: LotFinderDB.GetQueuedSearches component

The system SHALL retrieve all searches pending processing.

#### Method Signature

```csharp
Task<List<Search>> GetQueuedSearchesAsync(CancellationToken ct = default)
```

#### Query

```sql
SELECT s.ID, s.UserName, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT
FROM   dbo.Search s
WHERE  s.Status < 3
ORDER BY s.SubmitDT
```

#### Business Rules

- Status < 3 = New (0), Submitted (1), Started (2)
- Throws `QueryException` on error
- Ordered by SubmitDT ascending (FIFO processing)

#### Scenario: Worker polls for pending searches
- **WHEN** 3 searches with Status = Submitted exist and `GetQueuedSearchesAsync()` is called
- **THEN** 3 Search objects are returned ordered by SubmitDT (FIFO)

#### Scenario: Exclude completed searches
- **WHEN** 2 searches with Status = Ended (3) exist and `GetQueuedSearchesAsync()` is called
- **THEN** those searches are not included (Status < 3 filter)

---

### Requirement: LotFinderDB.GetSearch component

The system SHALL retrieve a single search by ID with criteria.

#### Method Signature

```csharp
Task<Search?> GetSearchAsync(int id, CancellationToken ct = default)
```

#### Query

```sql
SELECT s.UserName, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT,
       s.Criteria as CriteriaJSON
FROM   dbo.Search s
WHERE  s.ID = @id
```

#### Business Rules

- Returns null if not found
- Throws `QueryException` on database error
- Deserializes CriteriaJSON to SearchCriteria object
- Sets ID property manually after query (not in SELECT)

---

### Requirement: LotFinderDB.GetSearchResults component

The system SHALL retrieve Excel results for a completed search.

#### Method Signature

```csharp
Task<byte[]?> GetSearchResultsAsync(int id, CancellationToken ct = default)
```

#### Query

```sql
SELECT s.Results
FROM   dbo.Search AS s
WHERE  s.ID = @id
```

#### Business Rules

- Returns null if not found
- Throws `QueryException` on database error
- Results is VARBINARY(MAX) containing Excel file

---

### Requirement: LotFinderDB.SubmitSearch component

The system SHALL create a new search request.

#### Method Signature

```csharp
Task<int> SubmitSearchAsync(Search search, CancellationToken ct = default)
```

#### Stored Procedure

```sql
EXEC dbo.usp_SubmitSearch
    @p_UserName = ...,
    @p_Name = ...,
    @p_Criteria = ...,
    @o_SearchID = ... OUTPUT
```

#### Business Rules

- Sets Status = Submitted and SubmitDT = DateTime.UtcNow before insert
- Serializes Criteria to JSON via ToJSON() method
- Throws `QueryException` on error
- Uses stored procedure with OUTPUT parameter for new ID

---

### Requirement: LotFinderDB.SearchItems component

The system SHALL provide autocomplete search for items by number or description.

#### Method Signature

```csharp
Task<List<Item>> SearchItemsAsync(string filter, CancellationToken ct = default)
```

#### Query

```sql
SELECT TOP 25 i.ShortItemNumber, i.ItemNumber, i.Description, i.LastUpdateDT
FROM   dbo.Item AS i
WHERE  i.ItemNumber LIKE '%' + @filter + '%' OR
       i.Description LIKE '%' + @filter + '%'
ORDER BY i.ItemNumber
```

#### Business Rules

- Returns top 25 matches
- Case-insensitive LIKE search on both ItemNumber and Description
- Throws `QueryException` on error

---

### Requirement: LotFinderDB.LookupItems component

The system SHALL provide batch lookup of items by exact item numbers.

#### Method Signature

```csharp
Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default)
```

#### Query

```sql
SELECT i.ShortItemNumber, i.ItemNumber, i.Description, i.LastUpdateDT
FROM   dbo.Item AS i
INNER JOIN @itemNumbers AS i2 ON (i.ItemNumber = i2.ItemNumber)
ORDER BY i.ItemNumber
```

#### Business Rules

- Uses table-valued parameter `ItemNumberFilterParameter`
- Throws `QueryException` on error

---

### Requirement: LotFinderDB.LookupWorkorders component

The system SHALL provide batch lookup of work orders by work order numbers.

#### Method Signature

```csharp
Task<List<WorkOrder>> LookupWorkordersAsync(List<long> workorderNumbers, CancellationToken ct = default)
```

#### Query

```sql
SELECT *
FROM   dbo.WorkOrder AS wo
INNER JOIN @workOrderNumbers wo2 ON (wo.WorkOrderNumber = wo2.WorkOrderNumber)
```

#### Business Rules

- Uses table-valued parameter `WorkOrderFilterParameter`
- SELECT * returns all columns from WorkOrder view
- Throws `QueryException` on error

---

### Requirement: LotFinderDB.SearchWorkCenters / LookupWorkCenters component

The system SHALL provide autocomplete and batch lookup for work centers.

#### SearchWorkCenters Query

```sql
SELECT TOP 25 wc.Code, wc.Description, wc.LastUpdateDT
FROM   dbo.WorkCenter AS wc
WHERE  wc.Code LIKE '%' + @filter + '%' OR
       wc.Description LIKE '%' + @filter + '%'
ORDER BY wc.Code
```

#### LookupWorkCenters Query

```sql
SELECT wc.Code, wc.Description, wc.LastUpdateDT
FROM   dbo.WorkCenter AS wc
INNER JOIN @workCenterCodes wc2 ON (wc.Code = wc2.Code)
ORDER BY wc.Code
```

---

### Requirement: LotFinderDB.SearchProfitCenters / LookupProfitCenters component

The system SHALL provide autocomplete and batch lookup for profit centers.

#### SearchProfitCenters Query

```sql
SELECT TOP 25 pc.Code, pc.Description, pc.LastUpdateDT
FROM   dbo.ProfitCenter AS pc
WHERE  pc.Code LIKE '%' + @filter + '%' OR
       pc.Description LIKE '%' + @filter + '%'
ORDER BY pc.Code
```

#### LookupProfitCenters Query

```sql
SELECT pc.Code, pc.Description, pc.LastUpdateDT
FROM   dbo.ProfitCenter AS pc
INNER JOIN @profitCenterCodes AS pc2 ON (pc.Code = pc2.Code)
ORDER BY pc.Code
```

---

### Requirement: LotFinderDB.SearchUsers / LookupUsers component

The system SHALL provide autocomplete and batch lookup for JDE users.

#### SearchUsers Query

```sql
SELECT TOP 25 u.AddressNumber, COALESCE(u.UserID,' ') AS UserID, u.FullName, u.LastUpdateDT
FROM   dbo.JdeUser AS u
WHERE  u.UserID LIKE '%' + @filter + '%' OR
       u.FullName LIKE '%' + @filter + '%' OR
       CAST(u.AddressNumber AS VARCHAR(10)) LIKE '%' + @filter + '%'
ORDER BY u.UserID, u.FullName
```

#### LookupUsers Query

```sql
SELECT u.AddressNumber, u.UserID, u.FullName, u.LastUpdateDT
FROM   dbo.JdeUser AS u
INNER JOIN @userIDs u2 ON (u.UserID = u2.UserName OR CAST(u.AddressNumber AS VARCHAR(20)) = u2.UserName)
ORDER BY u.UserID
```

#### Business Rules

- SearchUsers: COALESCE UserID to space if null
- LookupUsers: Matches by UserID OR AddressNumber (cast to string)
- Uses table-valued parameter `OperatorFilterParameter`

---

### Requirement: LotFinderDB.LookupLots component

The system SHALL provide batch lookup of lots by lot number and item number.

#### Method Signature

```csharp
Task<List<Lot>> LookupLotsAsync(List<LotViewModel> lots, CancellationToken ct = default)
```

#### Query

```sql
SELECT DISTINCT l.LotNumber, l.BranchCode, l.ShortItemNumber, l.ItemNumber,
       l.SupplierCode, l.LastUpdateDT
FROM   dbo.Lot AS l
INNER JOIN @lotNumbers ln ON (l.LotNumber = ln.ComponentLotNumber AND
       ((l.ItemNumber IS NULL AND ln.ItemNumber IS NULL) OR l.ItemNumber = ln.ItemNumber))
```

#### Business Rules

- Uses table-valued parameter `ComponentLotFilterParameter`
- Matches on LotNumber AND ItemNumber (with NULL handling)
- DISTINCT to eliminate duplicates

---

### Requirement: LotFinderDB.PostProcessMisData component

The system SHALL post-process imported MIS data to set obsolete dates.

#### Method Signature

```csharp
Task PostProcessMisDataAsync(CancellationToken ct = default)
```

#### Query

```sql
SET ANSI_WARNINGS OFF;

-- Set ObsoleteDate for Current status when BackLevel exists
WITH cte AS (
  SELECT md.MisNumber, md.RevID, md.Status, MIN(md.ReleaseDate) Released
  FROM dbo.MisData AS md
  GROUP BY md.MisNumber, md.RevID, md.Status
)
UPDATE dbo.MisData
SET    ObsoleteDate = bl.Released
FROM   cte bl
WHERE  MisData.MisNumber = bl.MisNumber AND
       MisData.RevID = bl.RevID AND
       MisData.Status = 'Current' AND
       bl.Status = 'BackLevel';

-- Set ObsoleteDate for remaining records based on next revision
WITH cte AS (
  SELECT md.MisNumber, md.RevID, md.Status, MIN(md.ReleaseDate) Released
  FROM dbo.MisData AS md
  GROUP BY md.MisNumber, md.RevID, md.Status
)
UPDATE dbo.MisData
SET    ObsoleteDate = (SELECT TOP 1 nl.Released
                       FROM   cte nl
                       WHERE  MisData.MisNumber = nl.MisNumber AND
                              MisData.RevID < nl.RevID AND
                              MisData.Status = nl.Status
                       ORDER BY nl.RevID)
WHERE ObsoleteDate IS NULL;

ALTER INDEX [PK_MisData] ON [dbo].[MisData] REBUILD;
```

#### Business Rules

- Two-phase update: first BackLevel, then next revision
- Rebuilds primary key index after updates
- Disables ANSI_WARNINGS for aggregate operations

---

### Requirement: LotFinderDB.GetLastDataUpdates component

The system SHALL retrieve most recent successful update per table/type.

#### Method Signature

```csharp
Task<List<DataUpdate>> GetLastDataUpdatesAsync(CancellationToken ct = default)
```

#### Query

```sql
WITH DU_CTE AS (
  SELECT du.*,
         ROW_NUMBER() OVER (PARTITION BY du.TableName, du.UpdateType ORDER BY du.StartDT DESC) RN
  FROM   dbo.DataUpdate AS du
)
SELECT cte.SourceSystem, cte.SourceData, cte.TableName, cte.StartDT, cte.EndDT,
       cte.UpdateType, cte.WasSuccessful, cte.NumberRecords
FROM   DU_CTE cte
WHERE  cte.RN = 1
```

#### Business Rules

- Returns latest record per (TableName, UpdateType) combination
- Used to determine incremental update windows

---

### Requirement: LotFinderDB.GetTableSpec component

The system SHALL retrieve table schema for dynamic operations.

#### Method Signature

```csharp
Task<TableSpec> GetTableSpecAsync(string tableName, CancellationToken ct = default)
```

#### Queries

```sql
-- Get columns
SELECT c.name AS Name,
       CASE t2.name
         WHEN 'varchar' THEN 'VARCHAR(' + CAST(c.max_length AS VARCHAR(10)) + ')'
         WHEN 'decimal' THEN 'DECIMAL(' + CAST(c.precision AS VARCHAR(4)) + ',' + CAST(c.scale AS VARCHAR(4)) + ')'
         ELSE UPPER(t2.name)
       END AS Definition
FROM   sys.columns c
INNER JOIN sys.types AS t2 ON (c.system_type_id = t2.system_type_id)
INNER JOIN sys.tables t ON (c.object_id = t.object_id)
WHERE  t.name = @name
ORDER BY c.column_id

-- Get primary key columns
SELECT COLUMN_NAME AS Name
FROM   INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE  OBJECTPROPERTY(OBJECT_ID(CONSTRAINT_SCHEMA + '.' + QUOTENAME(CONSTRAINT_NAME)), 'IsPrimaryKey') = 1 AND
       TABLE_NAME = @name
ORDER BY ORDINAL_POSITION
```

#### Business Rules

- Used for dynamic temp table creation during bulk operations
- Maps SQL Server types to DDL definitions

---

### Requirement: LotFinderDB.RebuildIndices component

The system SHALL rebuild all indices on a table with SQL injection protection.

#### Method Signature

```csharp
Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)
```

#### Query

```sql
ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95);
```

#### Valid Table Names (Whitelist)

The following table names are valid for index rebuilding:

- `Branch`
- `DataUpdate`
- `FunctionCode`
- `Item`
- `JdeUser`
- `Lot`
- `LotLocation`
- `LotUsage_Curr`
- `LotUsage_Hist`
- `MisData`
- `OrgHierarchy`
- `ProfitCenter`
- `RouteMaster`
- `Search`
- `StatusCode`
- `WorkCenter`
- `WorkOrder_Curr`
- `WorkOrder_Hist`
- `WorkOrderComponent_Curr`
- `WorkOrderComponent_Hist`
- `WorkOrderRouting`
- `WorkOrderStep_Curr`
- `WorkOrderStep_Hist`
- `WorkOrderTime_Curr`
- `WorkOrderTime_Hist`

#### Business Rules

- Table name MUST be validated against whitelist before execution
- Throws `ArgumentException` if table name is not in whitelist
- FILLFACTOR = 95 leaves 5% free space for inserts
- Uses `DataAccessOptions.RebuildIndexTimeoutSeconds` for timeout

#### Scenario: Valid table name
- **WHEN** `RebuildIndicesAsync("WorkOrder_Curr")` is called
- **THEN** index rebuild executes successfully because table name is in whitelist

#### Scenario: Invalid table name (SQL injection attempt)
- **WHEN** `RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--")` is called
- **THEN** `ArgumentException` is thrown with message "Invalid table name"

---

### Requirement: LotFinderDB.GenerateTableParameter component

The system SHALL create DataTable for table-valued parameters.

#### Method Signature

```csharp
DataTable GenerateTableParameter<T>(List<T> keys)
```

#### Business Rules

- Creates single-column DataTable with column named "Key"
- Column type matches generic type T
- Used for simple single-value TVPs

---

## Domain Model Cross-Reference

| Repository Method | Domain Model | Cache Table |
|-------------------|--------------|-------------|
| JDE.GetWorkOrders | WorkOrder | WorkOrder_Curr/WorkOrder_Hist |
| JDE.GetWorkOrderSteps | WorkOrderStep | WorkOrderStep_Curr/WorkOrderStep_Hist |
| JDE.GetWorkOrderTimes | WorkOrderTime | WorkOrderTime_Curr/WorkOrderTime_Hist |
| JDE.GetWorkOrderRoutings | WorkOrderRouting | WorkOrderRouting |
| JDE.GetWorkOrderComponents | WorkOrderComponent | WorkOrderComponent_Curr/WorkOrderComponent_Hist |
| JDE.GetLots | Lot | Lot |
| JDE.GetLotUsages | LotUsage | LotUsage_Curr/LotUsage_Hist |
| JDE.GetLotLocations | LotLocation | LotLocation |
| JDE.GetItems | Item | Item |
| JDE.GetUsers | JdeUser | JdeUser |
| JDE.GetBranches | Branch | Branch |
| JDE.GetProfitCenters | ProfitCenter | ProfitCenter |
| JDE.GetWorkCenters | WorkCenter | WorkCenter |
| JDE.GetStatusCodes | StatusCode | StatusCode |
| JDE.GetFunctionCodes | FunctionCode | FunctionCode |
| JDE.GetOrgHierarchy | OrgHierarchy | OrgHierarchy |
| JDE.GetRouteMasters | RouteMaster | RouteMaster |
| CMS.GetMisData | MisData | MisData |
| LotFinderDB.GetSearch | Search | Search |
| LotFinderDB.GetLastDataUpdates | DataUpdate | DataUpdate |

---

## JDE Table Reference

| JDE Table | Schema | Description |
|-----------|--------|-------------|
| F4801 | PRODDTA / ARCDTAPD | Work Order Master |
| F3112 | PRODDTA / ARCDTAPD | Work Order Routing (Steps) |
| F31122 | PRODDTA / ARCDTAPD | Work Order Time Transactions |
| F3112Z1 | PRODDTA | Work Order Routing Transactions |
| F3111 | PRODDTA / ARCDTAPD | Work Order Parts List (Components) |
| F4108 | PRODDTA | Lot Master |
| F4111 | PRODDTA / ARCDTAPD | Item Ledger (Cardex) |
| F41021 | JDESTAGE | Item Location |
| F4101 | PRODDTA | Item Master |
| F0101 | PRODDTA | Address Book Master |
| F0092 | SY920 | User Profile |
| F0006 | PRODDTA | Business Unit Master |
| F0005 | JDESTAGE | User Defined Codes |
| F00192 | PRODDTA | Media Object (Function Codes) |
| F30006 | PRODDTA | Work Center Master |
| F3003 | PRODDTA | Routing Master |

---

## Migration Notes

| Legacy Pattern | New Pattern | Rationale |
|----------------|-------------|-----------|
| `System.Data.SqlClient` | `Microsoft.Data.SqlClient` | Modern SQL Server client with better performance and security |
| `Oracle.ManagedDataAccess.Client` + `DDTek.Oracle` | `Oracle.ManagedDataAccess.Core` | Consolidated to single Oracle driver for .NET Core/.NET 5+ |
| Static partial classes | Instance-based repositories with interfaces | Testability, dependency injection, loose coupling |
| External .sql files | Embedded resources or `const string` fields | Deployment simplicity, no runtime file I/O |
| `EncryptionHelper.Decrypt` with hardcoded key | Azure Key Vault or .NET Secret Manager | Industry standard secrets management |
| `buffered: false` Dapper | `QueryUnbufferedAsync` with `IAsyncEnumerable<T>` | Native async streaming, memory efficiency |
| `string.Format` SQL (RebuildIndices) | Table name whitelist validation | SQL injection prevention |
| Synchronous methods | Async/await with `CancellationToken` | Scalability, responsiveness, cancellation support |
| NLog static logger | `ILogger<T>` injected + `BeginScope()` for context | Modern structured logging with DI |
| Exception wrapping with generic message | Custom typed exceptions (`DataAccessException`, `ConnectionException`, `QueryException`, `DataAccessTimeoutException`) | Better error handling, debugging, and error classification |
| `Config.*CS` static properties | `IConfiguration["ConnectionStrings:*"]` | Configuration abstraction, environment support |
| Hardcoded timeout constants | `IOptions<DataAccessOptions>` with configurable values | Centralized configuration, runtime flexibility |
| DataTable for TVPs | Keep DataTable (Dapper compatible) | Dapper TVP support, no breaking change needed |
| Return null/empty on error | Throw typed exceptions | Consistent error handling, explicit failure semantics |

---

## Resolved Design Decisions

The following questions from the legacy analysis have been resolved with architectural decisions:

1. **Oracle driver consolidation**: Consolidated to `Oracle.ManagedDataAccess.Core` for all Oracle connections (JDE, JDE Stage, CMS). Single driver simplifies deployment and maintenance.

2. **External SQL files vs embedded resources**: SQL queries stored as embedded resources or compile-time constants. Eliminates runtime file I/O and simplifies deployment.

3. **Password encryption**: Replaced custom `EncryptionHelper.Decrypt` with Azure Key Vault or .NET Secret Manager. Industry-standard approach with proper key management.

4. **Streaming vs buffered**: All large dataset queries use `IAsyncEnumerable<T>` with `QueryUnbufferedAsync`. Streaming pattern preserved for memory efficiency.

5. **SQL injection in RebuildIndices**: Implemented table name whitelist validation. Only explicitly listed table names are allowed.

6. **Error handling inconsistency**: Standardized on throwing typed exceptions. No methods return null/empty on error conditions.

7. **GetUsers incremental bug**: Documented as intentional behavior (full sync always for users). No incremental filtering parameter.

8. **Special timeouts**: Made configurable via `DataAccessOptions`. LotUsage and MisData have separate configurable timeout values.

9. **Schema placeholders**: Schema names (`ProductionSchema`, `ArchiveSchema`, `StageSchema`) are configurable in `DataAccessOptions` for environment-specific deployments.

10. **StatusCode/LotLocation connection**: These queries use `CreateJdeStageConnectionAsync()` for JDESTAGE schema views. Separate connection configuration allows different credentials if needed.

11. **CMS connection visibility**: All connection methods are on `IDbConnectionFactory` interface, providing consistent public access pattern.

12. **Async support**: Full async support with `Task<T>` and `IAsyncEnumerable<T>` return types. All methods accept `CancellationToken`.

13. **Connection pooling**: Connection pooling is configured at the connection string level. `IDbConnectionFactory` opens connections; callers are responsible for disposal.

14. **Transaction support**: Transactions handled at the service layer when needed. Repository methods are single-unit operations.

15. **F41021_VIEW vs F41021**: JDESTAGE views provide flattened/optimized access. Original design preserved; uses JDE Stage connection.

16. **Archive methods**: Explicitly defined as separate interface methods (`GetWorkOrdersArchiveAsync`, etc.) for clarity.

17. **_FILTERED query variants**: Filtered queries use same method with optional `lastUpdateDT` parameter. Null = full sync, non-null = incremental.

---

## Codex Review Findings

The following observations were noted during legacy code analysis:

1. **Mixed driver usage**: Legacy code uses both `Oracle.ManagedDataAccess` and `DDTek.Oracle`. New design consolidates to single driver.

2. **Password decryption location**: Legacy decrypts in `Config.*CS` properties. New design retrieves secrets from secure store at connection time.

3. **LotFinderDB timeout comment**: Legacy comment says "ms" but value is seconds. New design uses explicit `TimeoutSeconds` naming.

4. **Query repository SetQuery unused**: Legacy `SetQuery` method writes to disk but appears unused. Not carried forward to new design.

5. **Inconsistent error patterns**: Some legacy methods return empty/null on error, others throw. New design standardizes on exceptions.