dohertj2/jdescopingtool

Fork 0

Files

T

Joseph Doherty 896ce59d8e docs: update specs and architecture docs with usp_/fn_ naming convention

2026-01-06 16:07:34 -05:00

73 KiB

Raw Blame History

Data Access Specification

Purpose

The data access layer provides interface-based repositories with dependency injection for accessing three distinct data sources in the JDE Scoping Tool application:

JDE Oracle - Primary enterprise system (JD Edwards) for manufacturing data
CMS (Oracle) - Manufacturing Information System (MIS) data (legacy used DDTek.Oracle driver, migrating to Oracle.ManagedDataAccess.Core)
SQL Server Cache (LotFinderDB) - Local cache database for search operations

The layer follows a modern repository pattern with:

Interface-based repositories (ILotFinderRepository, IJdeRepository, ICmsRepository) for testability and loose coupling
Connection factory abstraction (IDbConnectionFactory) for database connections
Dependency injection via extension methods for service registration
Async-first design with IAsyncEnumerable<T> for streaming and CancellationToken support
Typed exception hierarchy for consistent error handling

Source Reference

Legacy Files	Purpose
OLD/DataModel/Process/LotFinderDB.cs	SQL Server cache - base class with connection management
OLD/DataModel/Process/LotFinderDB.SearchManagement.cs	SQL Server cache - search CRUD operations
OLD/DataModel/Process/LotFinderDB.Item.cs	SQL Server cache - item search/lookup
OLD/DataModel/Process/LotFinderDB.WorkOrder.cs	SQL Server cache - work order lookup
OLD/DataModel/Process/LotFinderDB.WorkCenter.cs	SQL Server cache - work center search/lookup
OLD/DataModel/Process/LotFinderDB.Lot.cs	SQL Server cache - lot lookup
OLD/DataModel/Process/LotFinderDB.ProfitCenter.cs	SQL Server cache - profit center search/lookup
OLD/DataModel/Process/LotFinderDB.User.cs	SQL Server cache - user search/lookup
OLD/DataModel/Process/LotFinderDB.MisData.cs	SQL Server cache - MIS data post-processing
OLD/DataModel/Process/JDE.cs	JDE Oracle - base class with connection management
OLD/DataModel/Process/JDE.WorkOrders.cs	JDE Oracle - work order queries
OLD/DataModel/Process/JDE.WorkOrderStep.cs	JDE Oracle - work order step queries
OLD/DataModel/Process/JDE.WorkOrderTime.cs	JDE Oracle - work order time transaction queries
OLD/DataModel/Process/JDE.WorkOrderRouting.cs	JDE Oracle - work order routing transaction queries
OLD/DataModel/Process/JDE.WorkOrderComponent.cs	JDE Oracle - work order component queries
OLD/DataModel/Process/JDE.Lots.cs	JDE Oracle - lot queries
OLD/DataModel/Process/JDE.LotUsage.cs	JDE Oracle - lot usage (cardex) queries
OLD/DataModel/Process/JDE.LotLocation.cs	JDE Oracle - lot location queries
OLD/DataModel/Process/JDE.Items.cs	JDE Oracle - item queries
OLD/DataModel/Process/JDE.Users.cs	JDE Oracle - user queries
OLD/DataModel/Process/JDE.BusinessUnits.cs	JDE Oracle - branch/profit center/work center queries
OLD/DataModel/Process/JDE.StatusCodes.cs	JDE Oracle - status code queries
OLD/DataModel/Process/JDE.FunctionCode.cs	JDE Oracle - function code queries
OLD/DataModel/Process/JDE.OrgHierarchy.cs	JDE Oracle - organization hierarchy queries
OLD/DataModel/Process/JDE.RouteMaster.cs	JDE Oracle - route master queries
OLD/DataModel/Process/CMS.cs	CMS - base class with connection management
OLD/DataModel/Process/CMS.MisData.cs	CMS - MIS data queries
OLD/DataModel/Process/QueryRepository.cs	External SQL query file management
OLD/DataModel/Config.cs	Connection string configuration

Requirements

Requirement: Repository Interface Pattern

The system SHALL define repository interfaces for all data access operations.

Interface Definitions

public interface ILotFinderRepository
{
    // Search Management
    Task<List<Search>> GetUserSearchesAsync(string userName, CancellationToken ct = default);
    Task<List<Search>> GetQueuedSearchesAsync(CancellationToken ct = default);
    Task<Search?> GetSearchAsync(int id, CancellationToken ct = default);
    Task<byte[]?> GetSearchResultsAsync(int id, CancellationToken ct = default);
    Task<int> SubmitSearchAsync(Search search, CancellationToken ct = default);
    Task UpdateSearchStatusAsync(int id, SearchStatus status, CancellationToken ct = default);
    Task UpdateSearchResultsAsync(int id, byte[] results, CancellationToken ct = default);

    // Reference Data Lookup
    Task<List<Item>> SearchItemsAsync(string filter, CancellationToken ct = default);
    Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default);
    Task<List<WorkOrder>> LookupWorkordersAsync(List<long> workorderNumbers, CancellationToken ct = default);
    Task<List<WorkCenter>> SearchWorkCentersAsync(string filter, CancellationToken ct = default);
    Task<List<WorkCenter>> LookupWorkCentersAsync(List<string> codes, CancellationToken ct = default);
    Task<List<ProfitCenter>> SearchProfitCentersAsync(string filter, CancellationToken ct = default);
    Task<List<ProfitCenter>> LookupProfitCentersAsync(List<string> codes, CancellationToken ct = default);
    Task<List<JdeUser>> SearchUsersAsync(string filter, CancellationToken ct = default);
    Task<List<JdeUser>> LookupUsersAsync(List<string> userIds, CancellationToken ct = default);
    Task<List<Lot>> LookupLotsAsync(List<LotViewModel> lots, CancellationToken ct = default);

    // Data Sync Operations
    Task<List<DataUpdate>> GetLastDataUpdatesAsync(CancellationToken ct = default);
    Task<TableSpec> GetTableSpecAsync(string tableName, CancellationToken ct = default);
    Task RebuildIndicesAsync(string tableName, CancellationToken ct = default);
    Task PostProcessMisDataAsync(CancellationToken ct = default);
    Task<int> BulkInsertAsync<T>(string tableName, IEnumerable<T> records, CancellationToken ct = default);
    Task TruncateTableAsync(string tableName, CancellationToken ct = default);
}

public interface IJdeRepository
{
    // Work Order Data
    IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrder> GetWorkOrdersArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderRouting> GetWorkOrderRoutingsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);

    // Lot Data
    IAsyncEnumerable<Lot> GetLotsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotUsage> GetLotUsagesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotUsage> GetLotUsagesArchiveAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<LotLocation> GetLotLocationsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);

    // Reference Data
    IAsyncEnumerable<Item> GetItemsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<JdeUser> GetUsersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<Branch> GetBranchesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<ProfitCenter> GetProfitCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<WorkCenter> GetWorkCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<StatusCode> GetStatusCodesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<FunctionCode> GetFunctionCodesAsync(CancellationToken ct = default);
    IAsyncEnumerable<OrgHierarchy> GetOrgHierarchyAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
    IAsyncEnumerable<RouteMaster> GetRouteMastersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
}

public interface ICmsRepository
{
    IAsyncEnumerable<MisData> GetMisDataAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default);
}

Business Rules

All interfaces SHALL be registered with the DI container as scoped services
Implementations SHALL accept IDbConnectionFactory via constructor injection
Implementations SHALL accept ILogger<T> via constructor injection
Implementations SHALL accept IOptions<DataAccessOptions> via constructor injection

Scenario: Repository injection

WHEN a service requires data access and requests ILotFinderRepository via DI
THEN the container provides a configured LotFinderRepository instance with connection factory, logger, and options injected

Requirement: Connection Factory Pattern

The system SHALL provide a connection factory abstraction for database connections.

Interface Definition

public interface IDbConnectionFactory
{
    Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateJdeConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateJdeStageConnectionAsync(CancellationToken ct = default);
    Task<OracleConnection> CreateCmsConnectionAsync(CancellationToken ct = default);
}

Business Rules

Connection factory SHALL use Microsoft.Data.SqlClient.SqlConnection for SQL Server
Connection factory SHALL use Oracle.ManagedDataAccess.Core.OracleConnection for all Oracle connections (JDE, JDE Stage, CMS)
Connection strings SHALL be retrieved from IConfiguration["ConnectionStrings:*"]
Secrets (passwords) SHALL be retrieved from Azure Key Vault or .NET Secret Manager (never stored in config files)
Connections SHALL be opened asynchronously before returning
Connection factory SHALL be registered as a singleton service

Scenario: Create SQL Server connection

WHEN CreateLotFinderConnectionAsync() is called
THEN a new SqlConnection is created with connection string from configuration, opened asynchronously, and returned

Scenario: Create Oracle connection

WHEN CreateJdeConnectionAsync() is called
THEN a new OracleConnection is created with connection string from configuration, opened asynchronously, and returned

Requirement: Service Registration

The system SHALL provide DI extension methods for service registration.

Method Signature

public static IServiceCollection AddDataAccess(this IServiceCollection services, IConfiguration configuration)

Business Rules

Extension method SHALL register IDbConnectionFactory as singleton
Extension method SHALL register ILotFinderRepository as scoped
Extension method SHALL register IJdeRepository as scoped
Extension method SHALL register ICmsRepository as scoped
Extension method SHALL bind DataAccessOptions from configuration section "DataAccess"

Scenario: Register data access services

WHEN services.AddDataAccess(configuration) is called during startup
THEN all repository interfaces and connection factory are registered with appropriate lifetimes

Requirement: Configuration Options

The system SHALL support configurable timeouts via IOptions<DataAccessOptions>.

Options Class

public class DataAccessOptions
{
    public int DefaultTimeoutSeconds { get; set; } = 600;
    public int LotUsageTimeoutSeconds { get; set; } = 999999;
    public int MisDataTimeoutSeconds { get; set; } = 60000;
    public int RebuildIndexTimeoutSeconds { get; set; } = 600;
    public string ProductionSchema { get; set; } = "PRODDTA";
    public string ArchiveSchema { get; set; } = "ARCDTAPD";
    public string StageSchema { get; set; } = "JDESTAGE";
}

Business Rules

Default timeout SHALL be 600 seconds (10 minutes) for general queries
LotUsage filtered queries SHALL use configurable timeout (default 999999 seconds)
MIS data queries SHALL use configurable timeout (default 60000 seconds)
Schema names SHALL be configurable for environment-specific deployments

Requirement: Connection factory pattern

The system SHALL implement IDbConnectionFactory to provide database connections via dependency injection.

Implementation Pattern

public class DbConnectionFactory : IDbConnectionFactory
{
    private readonly IConfiguration _configuration;
    private readonly ILogger<DbConnectionFactory> _logger;

    public DbConnectionFactory(IConfiguration configuration, ILogger<DbConnectionFactory> logger)
    {
        _configuration = configuration;
        _logger = logger;
    }

    public async Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default)
    {
        var connectionString = _configuration.GetConnectionString("LotFinderDB")
            ?? throw new ConnectionException("LotFinderDB connection string not configured", "LotFinderDB");

        var connection = new SqlConnection(connectionString);
        try
        {
            await connection.OpenAsync(ct);
            return connection;
        }
        catch (SqlException ex)
        {
            _logger.LogError(ex, "Failed to connect to LotFinderDB");
            await connection.DisposeAsync();
            throw new ConnectionException("LotFinderDB: failed to open connection to database.", "LotFinderDB", ex);
        }
    }
}

Business Rules

Connection factory SHALL be registered as singleton
Connections SHALL be opened asynchronously before returning
Callers SHALL dispose returned connections when finished
ConnectionException SHALL be thrown on connection failure with inner exception preserved

Scenario: Successful connection creation

WHEN valid connection string exists in configuration
AND CreateLotFinderConnectionAsync() is called
THEN a new SqlConnection is created, opened, and returned
AND the caller is responsible for disposal

Scenario: Connection failure handling

WHEN database is unreachable
AND CreateLotFinderConnectionAsync() is called
THEN error is logged with exception details
AND ConnectionException is thrown with descriptive message
AND inner exception is preserved for debugging

Requirement: Async streaming pattern

The system SHALL use IAsyncEnumerable<T> with Dapper's QueryUnbufferedAsync for streaming large result sets.

Implementation Pattern

public async IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(
    DateTime? lastUpdateDT = null,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateJdeConnectionAsync(ct);

    var sql = ApplySchemaPlaceholders(
        lastUpdateDT.HasValue
            ? JdeQueries.SQL_GET_WORKORDERS_FILTERED
            : JdeQueries.SQL_GET_WORKORDERS);

    var command = new CommandDefinition(
        sql,
        parameters: lastUpdateDT.HasValue
            ? new { dateUpdated = ToJdeDate(lastUpdateDT.Value), timeUpdated = ToJdeTime(lastUpdateDT.Value) }
            : null,
        commandTimeout: _options.Value.DefaultTimeoutSeconds,
        cancellationToken: ct);

    await foreach (var workOrder in connection.QueryUnbufferedAsync<WorkOrder>(command).WithCancellation(ct))
    {
        yield return workOrder;
    }
}

Business Rules

All JDE/CMS collection queries SHALL return IAsyncEnumerable<T>
Queries SHALL use QueryUnbufferedAsync to stream results
Methods SHALL accept CancellationToken with [EnumeratorCancellation] attribute
Cancellation SHALL be checked between row iterations via WithCancellation(ct)

Scenario: Stream large work order dataset

WHEN GetWorkOrdersAsync() is called for 1 million work orders
THEN results are streamed via IAsyncEnumerable<WorkOrder> one at a time
AND memory usage remains constant regardless of result set size
AND consumer can use await foreach syntax

Scenario: Cancel streaming operation

WHEN cancellation is requested during GetWorkOrdersAsync() iteration
THEN iteration stops after current row completes
AND OperationCanceledException is thrown to consumer
AND database connection is properly disposed

Requirement: Schema placeholder replacement

The system SHALL replace schema placeholders in SQL queries from DataAccessOptions.

Implementation Pattern

private string ApplySchemaPlaceholders(string sql)
{
    return sql
        .Replace("{ProductionSchema}", _options.Value.ProductionSchema)
        .Replace("{ArchiveSchema}", _options.Value.ArchiveSchema)
        .Replace("{StageSchema}", _options.Value.StageSchema);
}

Business Rules

Schema names SHALL be configurable for environment-specific deployments
Default values: PRODDTA (production), ARCDTAPD (archive), JDESTAGE (stage)
Replacement SHALL occur at query execution time

Scenario: Query uses production schema

WHEN SQL query contains {ProductionSchema}.F4801
AND DataAccessOptions.ProductionSchema is "PRODDTA"
THEN query is executed with PRODDTA.F4801

Requirement: Table name whitelist validation

The system SHALL validate table names against an explicit whitelist to prevent SQL injection in RebuildIndicesAsync.

Implementation Pattern

private static readonly HashSet<string> ValidTableNames = new(StringComparer.OrdinalIgnoreCase)
{
    "Branch", "DataUpdate", "FunctionCode", "Item", "JdeUser",
    "Lot", "LotLocation", "LotUsage_Curr", "LotUsage_Hist",
    "MisData", "OrgHierarchy", "ProfitCenter", "RouteMaster",
    "Search", "StatusCode", "WorkCenter",
    "WorkOrder_Curr", "WorkOrder_Hist",
    "WorkOrderComponent_Curr", "WorkOrderComponent_Hist",
    "WorkOrderRouting",
    "WorkOrderStep_Curr", "WorkOrderStep_Hist",
    "WorkOrderTime_Curr", "WorkOrderTime_Hist"
};

public async Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)
{
    if (!ValidTableNames.Contains(tableName))
    {
        throw new ArgumentException($"Invalid table name: {tableName}", nameof(tableName));
    }

    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
    var sql = $"ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95)";
    await connection.ExecuteAsync(sql, commandTimeout: _options.Value.RebuildIndexTimeoutSeconds);
}

Business Rules

Table name MUST be validated against explicit whitelist before execution
ArgumentException SHALL be thrown for invalid table names
Comparison SHALL be case-insensitive

Scenario: Valid table name accepted

WHEN RebuildIndicesAsync("WorkOrder_Curr") is called
THEN table name passes whitelist validation
AND index rebuild executes successfully

Scenario: SQL injection attempt blocked

WHEN RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--") is called
THEN table name fails whitelist validation
AND ArgumentException is thrown with message "Invalid table name"
AND no SQL is executed

Requirement: Exception logging with scope context

The system SHALL log exceptions with structured scope context before throwing.

Implementation Pattern

catch (OracleException ex)
{
    using (_logger.BeginScope(new Dictionary<string, object>
    {
        ["DataSource"] = "JDE",
        ["Operation"] = nameof(GetWorkOrdersAsync),
        ["QueryName"] = "SQL_GET_WORKORDERS"
    }))
    {
        _logger.LogError(ex, "Query execution failed");
    }
    throw new QueryException("Failed to execute work order query", "SQL_GET_WORKORDERS", ex);
}

Business Rules

All exceptions SHALL be logged at throw site
Log context SHALL include: DataSource, Operation, and QueryName where applicable
Inner exceptions SHALL be preserved in thrown exceptions
Structured logging SHALL enable log aggregation and analysis

Scenario: Query exception with full context

WHEN JDE Oracle query fails with OracleException
THEN error is logged with BeginScope containing DataSource, Operation, QueryName
AND QueryException is thrown with descriptive message and inner exception

Requirement: Search criteria extraction via SQL functions

The system SHALL use SQL extraction functions to retrieve filter criteria directly from the Search.Criteria JSON column.

Implementation Pattern

Search query building now uses SearchId to invoke extraction functions rather than passing filter values from C#:

public SearchQueryResult BuildSearchQuery(int searchId)
{
    // Query builder generates SQL that calls extraction functions
    // Example generated SQL fragment:
    // INSERT INTO #P_WorkOrders SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(@SearchId)
    // INSERT INTO #P_ItemNumbers SELECT ItemNumber FROM dbo.fn_GetSearchItemNumbers(@SearchId)

    return new SearchQueryResult(sql, new { SearchId = searchId });
}

Business Rules

Query builder SHALL accept only searchId parameter (not full criteria object)
SQL queries SHALL call extraction functions to populate temporary filter tables
Extraction functions handle JSON parsing and validation in SQL Server
Invalid JSON or missing criteria results in empty filter sets (no errors thrown)

Available Extraction Functions

Function	Returns
`fn_GetSearchMinimumDt`	DATETIME2 scalar
`fn_GetSearchMaximumDt`	DATETIME2 scalar
`fn_GetSearchExtractMisData`	BIT scalar
`fn_GetSearchWorkOrders`	WorkOrderNumber table
`fn_GetSearchItemNumbers`	ItemNumber table
`fn_GetSearchProfitCenters`	Code table
`fn_GetSearchWorkCenters`	Code table
`fn_GetSearchOperatorIDs`	OperatorID table
`fn_GetSearchComponentLots`	LotNumber, ItemNumber table
`fn_GetSearchPartOperations`	ItemNumber, OperationNumber, MisNumber, MisRevision table

Scenario: Build search query with extraction functions

WHEN BuildSearchQuery(123) is called for a search with work order filter
THEN generated SQL includes SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(123)
AND only the @SearchId parameter is passed to the query

Requirement: Table-valued parameter support for lookups

The system SHALL use DataTable for SQL Server table-valued parameters in reference data lookup methods.

Business Rules

TVPs are used for batch lookups (LookupItemsAsync, LookupWorkordersAsync, etc.)
TVPs are NOT used for search query execution (replaced by extraction functions)
DataTable column names SHALL match TVP type column names

Scenario: Batch lookup with TVP

WHEN LookupItemsAsync(["ITEM001", "ITEM002", "ITEM003"]) is called
THEN DataTable is created with ItemNumber column
AND single query executes with TVP parameter
AND matching items are returned

Requirement: Service registration extension method

The system SHALL provide AddDataAccess extension method for DI registration.

Implementation Pattern

public static IServiceCollection AddDataAccess(
    this IServiceCollection services,
    IConfiguration configuration)
{
    services.Configure<DataAccessOptions>(
        configuration.GetSection(DataAccessOptions.SectionName));

    services.AddSingleton<IDbConnectionFactory, DbConnectionFactory>();
    services.AddScoped<ILotFinderRepository, LotFinderRepository>();
    services.AddScoped<IJdeRepository, JdeRepository>();
    services.AddScoped<ICmsRepository, CmsRepository>();

    return services;
}

Business Rules

Extension method SHALL bind DataAccessOptions from "DataAccess" configuration section
Connection factory SHALL be registered as singleton
Repositories SHALL be registered as scoped services
Method SHALL return IServiceCollection for chaining

Scenario: Register all data access services

WHEN services.AddDataAccess(configuration) is called during startup
THEN DataAccessOptions is bound from configuration
AND IDbConnectionFactory is registered as singleton
AND all repository interfaces are registered as scoped

Exception Handling

Requirement: Custom Exception Types

The system SHALL define custom exception types for data access errors.

Exception Hierarchy

public class DataAccessException : Exception
{
    public string? Operation { get; }
    public string? Repository { get; }
    public DataAccessException(string message, string? operation = null, string? repository = null, Exception? inner = null);
}

public class ConnectionException : DataAccessException
{
    public string? DataSource { get; }
    public ConnectionException(string message, string dataSource, Exception? inner = null);
}

public class QueryException : DataAccessException
{
    public string? QueryName { get; }
    public QueryException(string message, string queryName, Exception? inner = null);
}

public class DataAccessTimeoutException : DataAccessException
{
    public int TimeoutSeconds { get; }
    public DataAccessTimeoutException(string message, int timeoutSeconds, Exception? inner = null);
}

Business Rules

All repository methods SHALL throw typed exceptions on error (never return null/empty on error)
ConnectionException SHALL be thrown for connection failures
QueryException SHALL be thrown for query execution failures
DataAccessTimeoutException SHALL be thrown for timeout errors
Exception SHALL be logged at throw site via ILogger<T> with BeginScope() for context
Inner exceptions SHALL be preserved for debugging

Scenario: Connection failure logging

WHEN a connection to JDE Oracle fails
THEN error is logged with scope context (data source, operation) and ConnectionException is thrown with descriptive message and inner exception

Scenario: Query timeout

WHEN a query exceeds configured timeout
THEN DataAccessTimeoutException is thrown with timeout value and query name

Query Management

Requirement: QueryRepository component

The system SHALL manage SQL queries as embedded resources or compile-time constants.

Business Rules

SQL queries SHALL be stored as embedded resources in the assembly or as const string fields
Queries SHALL be loaded once at application startup and cached in memory
Schema placeholder replacement (PRODDTA, ARCDTAPD, JDESTAGE) SHALL use values from DataAccessOptions
Query loading SHALL throw QueryException if query is not found

Scenario: Load query from embedded resource

WHEN repository is instantiated and SQL queries are needed
THEN queries are loaded from embedded resources with schema placeholders replaced from configuration

Scenario: Query not found

WHEN a requested query name does not exist in embedded resources
THEN QueryException is thrown with message containing the query name

Connection Management

Requirement: Connection Management - LotFinderDB component

The system SHALL provide SQL Server cache database connections via IDbConnectionFactory.

Method: CreateLotFinderConnectionAsync

Opens a new connection to the LotFinderDB SQL Server database.

Parameter	Type	Description
ct	CancellationToken	Cancellation token for async operation

Returns	Type	Description
connection	Task	Open connection to LotFinderDB

Business Rules

Connection string loaded from IConfiguration["ConnectionStrings:LotFinderDB"]
Password retrieved from Azure Key Vault or .NET Secret Manager
Uses Microsoft.Data.SqlClient.SqlConnection
Returns opened connection (caller must dispose)
Throws ConnectionException on failure with descriptive message
Default command timeout: configurable via DataAccessOptions.DefaultTimeoutSeconds

Scenario: Successful connection

WHEN valid LotFinderDB connection string exists in configuration and CreateLotFinderConnectionAsync() is called
THEN a new SqlConnection is created, opened asynchronously, and returned

Scenario: Connection failure

WHEN invalid or unreachable database exists and CreateLotFinderConnectionAsync() is called
THEN error is logged via ILogger<T> and ConnectionException is thrown with message "LotFinderDB: failed to open connection to database."

Requirement: Connection Management - JDE component

The system SHALL provide JDE Oracle database connections via IDbConnectionFactory.

Method: CreateJdeConnectionAsync

Opens a new connection to the JDE Oracle database.

Parameter	Type	Description
ct	CancellationToken	Cancellation token for async operation

Returns	Type	Description
connection	Task	Open connection to JDE (Oracle.ManagedDataAccess.Core)

Business Rules

Connection string loaded from IConfiguration["ConnectionStrings:JDE"]
Password retrieved from Azure Key Vault or .NET Secret Manager
Uses Oracle.ManagedDataAccess.Core.OracleConnection
Returns opened connection (caller must dispose)
Throws ConnectionException on failure

Scenario: Successful JDE connection

WHEN valid JDE Oracle connection string exists and CreateJdeConnectionAsync() is called
THEN a new OracleConnection is created, opened asynchronously, and returned

Requirement: Connection Management - JDE Stage component

The system SHALL provide a separate JDE connection for stage/view queries.

Method: CreateJdeStageConnectionAsync

Opens a new connection to the JDE Stage database (for JDESTAGE schema views).

Business Rules

Connection string loaded from IConfiguration["ConnectionStrings:JDEStage"]
Uses Oracle.ManagedDataAccess.Core.OracleConnection
Used for GetStatusCodes() and GetLotLocations() methods
Queries JDESTAGE schema views (F0005_VIEW, F41021_VIEW)

Requirement: Connection Management - CMS component

The system SHALL provide CMS database connections via IDbConnectionFactory.

Method: CreateCmsConnectionAsync

Opens a new connection to the CMS database.

Parameter	Type	Description
ct	CancellationToken	Cancellation token for async operation

Returns	Type	Description
connection	Task	Open connection to CMS

Business Rules

Connection string loaded from IConfiguration["ConnectionStrings:CMS"]
Password retrieved from Azure Key Vault or .NET Secret Manager
Uses Oracle.ManagedDataAccess.Core.OracleConnection (consolidated from legacy DDTek.Oracle)

Async Streaming

Requirement: IAsyncEnumerable Streaming

The system SHALL use IAsyncEnumerable<T> for streaming large datasets.

Business Rules

All JDE/CMS query methods returning collections SHALL return IAsyncEnumerable<T>
Implementations SHALL use Dapper's QueryUnbufferedAsync for streaming results
All streaming methods SHALL accept CancellationToken parameter
Cancellation SHALL be checked between row iterations

Scenario: Stream work orders

WHEN GetWorkOrdersAsync() is called with large dataset
THEN results are streamed via IAsyncEnumerable<WorkOrder> without loading all rows into memory

Scenario: Cancel streaming operation

WHEN cancellation is requested during GetWorkOrdersAsync() iteration
THEN iteration stops and OperationCanceledException is thrown

JDE Oracle Queries

Requirement: JDE.GetWorkOrders component

The system SHALL fetch work order records from JDE F4801 table.

Method Signature

IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Parameters

Parameter	Type	Description
lastUpdateDT	DateTime?	Optional cutoff for incremental sync
ct	CancellationToken	Cancellation token

Returns

IAsyncEnumerable<WorkOrder> - Streaming results via QueryUnbufferedAsync

Query: SQL_GET_WORKORDERS

SELECT wo.WADOCO AS WorkOrderNumber,
       TRIM(wo.WAMMCU) AS BranchCode,
       TRIM(wo.WALOTN) AS LotNumber,
       TRIM(wo.WALITM) AS ItemNumber,
       wo.WAITM AS ShortItemNumber,
       TRIM(wo.WAPARS) AS ParentWorkOrderNumber,
       wo.WAUORG / 100.0 AS OrderQuantity,
       wo.WASOBK / 100.0 AS HeldQuantity,
       wo.WASOQS / 100.0 AS ShippedQuantity,
       TRIM(wo.WASRST) AS StatusCode,
       CASE wo.WADCG WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WADCG+1900000,'YYYYDDD') END AS StatusCodeUpdateDT,
       CASE wo.WATRDJ WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WATRDJ+1900000,'YYYYDDD') END AS IssueDate,
       CASE wo.WASTRT WHEN 0 THEN TO_DATE('1900-01-01', 'YYYY-MM-DD')
            ELSE TO_DATE(wo.WASTRT+1900000,'YYYYDDD') END AS StartDate,
       TRIM(wo.WATRT) AS RoutingType,
       wo.WAUPMJ AS LastUpdateDate,
       wo.WATDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4801 wo

Query: SQL_GET_WORKORDERS_FILTERED (Incremental)

Same SELECT with WHERE clause:

WHERE (wo.WAUPMJ > :dateUpdated OR
       (wo.WAUPMJ = :dateUpdated AND wo.WATDAY >= :timeUpdated))

Query: SQL_GET_WORKORDERS_ARCHIVE

Same SELECT from {ArchiveSchema}.F4801 (archive schema)

JDE Table Mapping

JDE Column	Domain Property	Transformation
WADOCO	WorkOrderNumber	Direct
WAMMCU	BranchCode	TRIM
WALOTN	LotNumber	TRIM
WALITM	ItemNumber	TRIM
WAITM	ShortItemNumber	Direct
WAPARS	ParentWorkOrderNumber	TRIM
WAUORG	OrderQuantity	/ 100.0
WASOBK	HeldQuantity	/ 100.0
WASOQS	ShippedQuantity	/ 100.0
WASRST	StatusCode	TRIM
WADCG	StatusCodeUpdateDT	JDE date conversion
WATRDJ	IssueDate	JDE date conversion
WASTRT	StartDate	JDE date conversion
WATRT	RoutingType	TRIM
WAUPMJ	LastUpdateDate	JDE date (integer)
WATDAY	LastUpdateTime	JDE time (integer)

Business Rules

Quantities stored in JDE as integer * 100, divided for decimal values
JDE dates stored as integer CYYDDD format, converted to DATE
Zero dates converted to '1900-01-01'
Uses configurable query timeout from DataAccessOptions.DefaultTimeoutSeconds
Streaming results via QueryUnbufferedAsync for memory efficiency
Schema placeholder {ProductionSchema} replaced from configuration

Scenario: Full sync (mass update)

WHEN lastUpdateDT is null and GetWorkOrdersAsync() is called
THEN SQL_GET_WORKORDERS query returns all work orders from JDE as async stream

Scenario: Incremental sync (daily/hourly update)

WHEN lastUpdateDT is 2024-01-15 and GetWorkOrdersAsync(lastUpdateDT) is called
THEN SQL_GET_WORKORDERS_FILTERED returns only records updated since that date

Scenario: JDE quantity conversion

WHEN JDE WAUORG value is 10050 (integer) and mapping to WorkOrder.OrderQuantity
THEN OrderQuantity equals 100.50 (decimal, divided by 100)

Requirement: JDE.GetWorkOrderSteps component

The system SHALL fetch work order operation steps from JDE F3112 table.

Method Signature

IAsyncEnumerable<WorkOrderStep> GetWorkOrderStepsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_WORKORDER_STEP

SELECT wos.WLDOCO AS WorkOrderNumber,
       wos.WLOPSQ/10 AS StepNumber,
       TRIM(wos.WLMCU) AS WorkCenterCode,
       TRIM(wos.WLMMCU) AS BranchCode,
       TRIM(wos.WLDSC1) AS StepDescription,
       TRIM(mes.CFDS80) AS FunctionOperationDescription,
       wos.WLOPSC AS StepTypeCode,
       CASE wos.WLSTRT WHEN 0 THEN NULL
            ELSE TO_DATE(wos.WLSTRT+1900000,'YYYYDDD') END AS StartDT,
       CASE wos.WLSTRX WHEN 0 THEN NULL
            ELSE TO_DATE(wos.WLSTRX+1900000,'YYYYDDD') END AS EndDT,
       TRIM(wos.WLURRF) AS FunctionCode,
       wos.WLSOCN / 100.0 AS ScrappedQuantity,
       wos.WLUPMJ AS LastUpdateDate,
       wos.WLTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3112 wos LEFT OUTER JOIN
       {ProductionSchema}.F00192 mes ON (wos.WLURRF = mes.CFKY)
 WHERE TRIM(wos.WLMCU) IS NOT NULL AND
       TRIM(wos.WLMMCU) IS NOT NULL

Business Rules

StepNumber is WLOPSQ/10 (stored as tenths in JDE)
LEFT OUTER JOIN to F00192 for FunctionOperationDescription
Filters out records with NULL work center or branch
Archive queries use {ArchiveSchema}.F3112

Scenario: Step number conversion

WHEN JDE WLOPSQ value is 100 (tenths) and mapping to WorkOrderStep.StepNumber
THEN StepNumber equals 10.0 (decimal)

Scenario: Enrich with function description

WHEN step has FunctionCode "ASSY" and LEFT OUTER JOIN to F00192 succeeds
THEN FunctionOperationDescription is populated from media object

Requirement: JDE.GetWorkOrderTimes component

The system SHALL fetch work order time transactions (operator labor) from JDE F31122 table.

Method Signature

IAsyncEnumerable<WorkOrderTime> GetWorkOrderTimesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_WORKORDER_TIMES

SELECT wot.WTUKID AS UniqueID,
       wot.WTDOCO AS WorkOrderNumber,
       wot.WTOPSQ/10 AS StepNumber,
       TRIM(wot.WTMCU) AS WorkCenterCode,
       TRIM(wot.WTMMCU) AS BranchCode,
       wot.WTAN8 AS AddressNumber,
       CASE wot.WTDGL WHEN 0 THEN NULL
            ELSE TO_DATE(wot.WTDGL+1900000,'YYYYDDD') END AS GlDate,
       wot.WTUPMJ AS LastUpdateDate,
       wot.WTTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F31122 wot
 WHERE TRIM(wot.WTMCU) IS NOT NULL AND
       TRIM(wot.WTMMCU) IS NOT NULL

Business Rules

Links operators to work orders via AddressNumber -> JdeUser
Filters out records with NULL work center or branch

Requirement: JDE.GetWorkOrderRoutings component

The system SHALL fetch work order routing transactions from JDE F3112Z1 table.

Method Signature

IAsyncEnumerable<WorkOrderRouting> GetWorkOrderRoutingsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_WORKORDER_ROUTING

SELECT TRIM(woz.SZEDUS) AS UserID,
       TRIM(woz.SZEDBT) AS BatchNumber,
       TRIM(woz.SZEDTN) AS TransactionNumber,
       woz.SZEDLN AS LineNumber,
       woz.SZOPSQ / 10.0 AS StepNumber,
       TRIM(woz.SZMCU) AS WorkCenterCode,
       woz.SZDOCO AS WorkOrderNumber,
       TRIM(woz.SZTRT) AS RoutingType,
       TRIM(woz.SZMMCU) AS BranchCode,
       TRIM(woz.SZDSC1) AS StepDescription,
       TRIM(woz.SZURRF) AS FunctionCode,
       woz.SZTRDJ AS TransactionDate_Date,
       woz.SZUPMJ AS LastUpdateDate,
       woz.SZTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3112Z1 woz
 WHERE woz.SZTYTN = 'JDERTG' AND
       woz.SZDRIN = '2' AND
       woz.SZTNAC = '02' AND
       woz.SZPID = 'ER31410' AND
       TRIM(woz.SZEDUS) IS NOT NULL AND
       TRIM(woz.SZEDBT) IS NOT NULL AND
       TRIM(woz.SZEDTN) IS NOT NULL AND
       TRIM(woz.SZMCU) IS NOT NULL

Business Rules

Filters by specific transaction type (JDERTG), direction (2), action (02), program ID (ER31410)
Used for MIS matching to find original operation sequence
Data validation: Skips records where LastUpdateDT or TransactionDate year < 1900 or > 2500
No archive table (single table for all data)

Requirement: JDE.GetWorkOrderComponents component

The system SHALL fetch work order component usage from JDE F3111 table.

Method Signature

IAsyncEnumerable<WorkOrderComponent> GetWorkOrderComponentsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_WORKORDER_COMPONENTS

SELECT woc.WMUKID AS UniqueID,
       woc.WMDOCO AS WorkOrderNumber,
       TRIM(woc.WMLOTN) AS LotNumber,
       TRIM(woc.WMCMCU) AS BranchCode,
       woc.WMCPIT AS ShortItemNumber,
       woc.WMTRQT / 100.0 AS Quantity,
       woc.WMUPMJ AS LastUpdateDate,
       woc.WMTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3111 woc
 WHERE TRIM(woc.WMLOTN) IS NOT NULL

Business Rules

Filters records with NULL lot number
Archive queries use {ArchiveSchema}.F3111

Requirement: JDE.GetLots component

The system SHALL fetch lot master data from JDE F4108 table.

Method Signature

IAsyncEnumerable<Lot> GetLotsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_LOTS

SELECT TRIM(lot.IOLOTN) AS LotNumber,
       TRIM(lot.IOMCU) AS BranchCode,
       lot.IOITM AS ShortItemNumber,
       TRIM(lot.IOLITM) AS ItemNumber,
       lot.IOVEND AS SupplierCode,
       lot.IOLOTS AS StatusCode,
       lot.IOLOT1 AS Memo1,
       lot.IOLOT2 AS Memo2,
       lot.IOLOT3 AS Memo3,
       lot.IOUPMJ AS LastUpdateDate,
       lot.IOTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4108 lot
 WHERE TRIM(lot.IOLOTN) IS NOT NULL AND
       TRIM(lot.IOMCU) IS NOT NULL

Business Rules

StatusCode (IOLOTS) is single character
Filters out records with NULL lot number or branch code

Requirement: JDE.GetLotUsages component

The system SHALL fetch lot usage (cardex) transactions from JDE F4111 table.

Method Signature

IAsyncEnumerable<LotUsage> GetLotUsagesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_LOT_USAGES

SELECT lu.ILUKID AS UniqueID,
       lu.ILDOCO AS WorkOrderNumber,
       TRIM(lu.ILLOTN) AS LotNumber,
       TRIM(lu.ILMCU) AS BranchCode,
       lu.ILITM AS ShortItemNumber,
       lu.ILTRQT AS Quantity,
       lu.ILTRDJ AS LastUpdateDate,
       lu.ILTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4111 lu
 WHERE lu.ILDCT = 'IM' AND
       TRIM(lu.ILLOTN) IS NOT NULL

Business Rules

Filters by document type 'IM' (inventory management)
Filters out records with NULL lot number
Special timeout: Filtered query uses DataAccessOptions.LotUsageTimeoutSeconds (default 999999)
Archive queries use {ArchiveSchema}.F4111

Requirement: JDE.GetLotLocations component

The system SHALL fetch lot location tracking from JDESTAGE.F41021_VIEW.

Method Signature

IAsyncEnumerable<LotLocation> GetLotLocationsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_LOT_LOCATIONS

SELECT TRIM(il.LOT_LILOTN) AS LotNumber,
       il.IDENTIFIERSHORTITEM_LIITM AS ShortItemNumber,
       TRIM(il.COSTCENTER_LIMCU) AS BranchCode,
       COALESCE(TRIM(il.LOCATION_LILOCN), ' ') AS Location,
       il.DATEUPDATED_LIUPMJ + FLOOR(il.TIMEOFDAY_LITDAY / 10000) / 24 +
         FLOOR(MOD(il.TIMEOFDAY_LITDAY, 10000) / 100) / 1440 +
         MOD(il.TIMEOFDAY_LITDAY, 100) / 86400 AS LastUpdateDT
  FROM {StageSchema}.F41021_VIEW il
 WHERE TRIM(il.LOT_LILOTN) IS NOT NULL

Business Rules

Uses JDESTAGE view via CreateJdeStageConnectionAsync() (different connection than main JDE)
Location defaults to space if NULL
LastUpdateDT computed inline from JDE date/time integers

Requirement: JDE.GetItems component

The system SHALL fetch item (part number) master data from JDE F4101 table.

Method Signature

IAsyncEnumerable<Item> GetItemsAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_ITEMS

SELECT pn.IMITM AS ShortItemNumber,
       TRIM(pn.IMLITM) AS ItemNumber,
       TRIM(pn.IMDSC1) AS Description,
       TRIM(pn.IMPRP4) AS PlanningFamily,
       TRIM(pn.IMSTKT) AS StockingType,
       pn.IMUPMJ AS LastUpdateDate,
       pn.IMTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F4101 pn
 WHERE TRIM(pn.IMLITM) IS NOT NULL

Requirement: JDE.GetUsers component

The system SHALL fetch user/operator data from JDE F0101 and SY920.F0092 tables.

Method Signature

IAsyncEnumerable<JdeUser> GetUsersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_USERS

WITH USER_CTE AS (
  SELECT ab.ABAN8 AS AddressNumber,
         TRIM(pro.ULUSER) AS UserID,
         TRIM(ab.ABALPH) AS FullName,
         ab.ABUPMJ AS LastUpdateDate,
         ab.ABUPMT AS LastUpdateTime,
         ROW_NUMBER() OVER (PARTITION BY ab.ABAN8 ORDER BY ab.ABUPMJ DESC, ab.ABUPMT DESC) RN
   FROM  {ProductionSchema}.F0101 ab LEFT OUTER JOIN
         SY920.F0092 pro ON (ab.ABAN8 = pro.ULAN8)
)
SELECT AddressNumber,
       UserID,
       FullName,
       LastUpdateDate,
       LastUpdateTime
  FROM USER_CTE
 WHERE RN = 1

Business Rules

CTE with ROW_NUMBER to get latest record per AddressNumber
LEFT OUTER JOIN to SY920 schema for UserID lookup
Note: Incremental filtering not supported for users (full sync always)

Requirement: JDE.GetBranches / GetProfitCenters / GetWorkCenters component

The system SHALL fetch business unit reference data from JDE F0006 table.

Method Signature

IAsyncEnumerable<Branch> GetBranchesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
IAsyncEnumerable<ProfitCenter> GetProfitCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)
IAsyncEnumerable<WorkCenter> GetWorkCentersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_BUSINESS_UNITS

SELECT TRIM(wc.MCMCU) AS Code,
       TRIM(wc.MCDL01) AS Description,
       wc.MCUPMJ AS LastUpdateDate,
       wc.MCUPMT AS LastUpdateTime
  FROM {ProductionSchema}.F0006 wc
 WHERE wc.MCSTYL = :typeCode

Parameters

Entity	typeCode Parameter
Branch	'BP'
ProfitCenter	'I3'
WorkCenter	'WC'

Requirement: JDE.GetStatusCodes component

The system SHALL fetch work order status codes from JDESTAGE.F0005_VIEW.

Method Signature

IAsyncEnumerable<StatusCode> GetStatusCodesAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_STATUS_CODES

SELECT TRIM(sc.USERDEFINEDCODE_DRKY) AS CODE,
       TRIM(sc.DESCRIPTION001_DRDL01) AS Description,
       sc.DATEUPDATED_DRUPMJ + FLOOR(sc.TIMELASTUPDATED_DRUPMT / 10000) / 24 +
         FLOOR(MOD(sc.TIMELASTUPDATED_DRUPMT, 10000) / 100) / 1440 +
         MOD(sc.TIMELASTUPDATED_DRUPMT, 100) / 86400 AS LastUpdateDT
  FROM {StageSchema}.F0005_VIEW sc
 WHERE TRIM(sc.PRODUCTCODE_DRSY) = '00' AND
       sc.USERDEFINEDCODES_DRRT = 'SS' AND
       TRIM(sc.USERDEFINEDCODE_DRKY) IS NOT NULL

Business Rules

Uses JDE Stage connection via CreateJdeStageConnectionAsync() (different connection than main JDE)
Filters by product code '00' and UDC type 'SS'
LastUpdateDT computed inline from date/time integers

Requirement: JDE.GetFunctionCodes component

The system SHALL fetch function code lookup data from JDE F00192 table.

Method Signature

IAsyncEnumerable<FunctionCode> GetFunctionCodesAsync(CancellationToken ct = default)

Query: SQL_GET_FUNCTION_CODES

SELECT Code,
       TRIM(LISTAGG(Description, ' ') WITHIN GROUP(ORDER BY Description) ||
         CASE WHEN MAX(total_lengthb) > 4000 THEN '...' ELSE '' END) Description,
       SYSDATE AS LastUpdateDT
FROM (
  SELECT TRIM(fc.CFKY) AS Code,
         TRIM(ASCIISTR(fc.CFDS80)) AS Description,
         SUM(LENGTHB(TRIM(fc.CFDS80))+1) OVER(PARTITION BY TRIM(fc.CFKY)
           ORDER BY TRIM(fc.CFDS80)) - 1 cumul_lengthb,
         SUM(LENGTHB(TRIM(fc.CFDS80))+1) OVER(PARTITION BY TRIM(fc.CFKY)) - 1 total_lengthb,
         COUNT(*) OVER(PARTITION BY TRIM(fc.CFKY)) num_values
  FROM   {ProductionSchema}.F00192 fc
  WHERE  TRIM(fc.CFKY) IS NOT NULL
)
WHERE total_lengthb <= 4000 OR cumul_lengthb <= 4000 - length('...')
GROUP BY Code

Business Rules

Aggregates multiple description rows per code using LISTAGG
Truncates at 4000 bytes with '...' suffix
Uses ASCIISTR to handle special characters
LastUpdateDT set to SYSDATE (current timestamp)
Note: Does not support incremental filtering (no lastUpdateDT parameter)

Requirement: JDE.GetOrgHierarchy component

The system SHALL fetch organization hierarchy (work center to profit center mapping) from JDE F30006 table.

Method Signature

IAsyncEnumerable<OrgHierarchy> GetOrgHierarchyAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_ORG_HIERARCHY

SELECT TRIM(oh.IWMCUW) AS ProfitCenterCode,
       TRIM(oh.IWMCU) AS WorkCenterCode,
       TRIM(oh.IWMMCU) AS BranchCode,
       oh.IWUPMJ AS LastUpdateDate,
       oh.IWTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F30006 oh
 WHERE TRIM(oh.IWMCU) IS NOT NULL AND
       TRIM(oh.IWMMCU) IS NOT NULL

Requirement: JDE.GetRouteMasters component

The system SHALL fetch item routing master data from JDE F3003 table.

Method Signature

IAsyncEnumerable<RouteMaster> GetRouteMastersAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_ROUTE_MASTER

SELECT TRIM(rm.IRMMCU) AS BranchCode,
       TRIM(rm.IRKITL) AS ItemNumber,
       TRIM(rm.IRTRT) AS RoutingType,
       rm.IROPSQ / 10.0 AS SequenceNumber,
       TRIM(rm.IRURRF) AS FunctionCode,
       TRIM(rm.IRMCU) AS WorkCenterCode,
       rm.IREFFF AS StartDate_Date,
       rm.IREFFT AS EndDate_Date,
       rm.IRUPMJ AS LastUpdateDate,
       rm.IRTDAY AS LastUpdateTime
  FROM {ProductionSchema}.F3003 rm
 WHERE TRIM(rm.IRKITL) IS NOT NULL

Business Rules

SequenceNumber is IROPSQ/10 (stored as tenths)
StartDate_Date and EndDate_Date are JDE integer format (converted in model)

CMS Queries

Requirement: CMS.GetMisData component

The system SHALL fetch Manufacturing Information System (MIS) data from CMS database.

Method Signature

IAsyncEnumerable<MisData> GetMisDataAsync(DateTime? lastUpdateDT = null, CancellationToken ct = default)

Query: SQL_GET_MIS_DATA

SELECT DISTINCT
       mis.P_PART_NUMBER AS ItemNumber,
       mis.P_OPERATION_NUMBER AS SequenceNumber,
       item.PITEM_ID AS MISNumber,
       itemrev.PITEM_REVISION_ID AS RevID,
       TRIM(mis.P_SITE) AS BranchCode,
       zim_test_details.P_SEQ_NUMBER AS CharNumber,
       zim_test_details.P_TEST_DESC AS TestDescription,
       zim_test_details.P_SAMPL_TYPE AS SamplingType,
       zim_test_details.P_SAMPL_VALUE AS SamplingValue,
       zim_test_details.P_TOOLS AS ToolsGauges,
       zim_test_details.P_WORK_INTR AS WorkInstructions,
       Status.PNAME AS Status,
       Status.PDATE_RELEASED AS ReleaseDate
  FROM INFODBA.PITEM item
       INNER JOIN INFODBA.PITEMREVISION itemrev ON (item.PUID = itemrev.RITEMS_TAGU)
       INNER JOIN INFODBA.PRELEASE_STATUS_LIST listing ON (itemrev.PUID = listing.PUID)
       INNER JOIN INFODBA.PRELEASESTATUS Status ON (listing.PVALU_0 = Status.PUID)
       INNER JOIN INFODBA.PIMANRELATION imanrel ON (itemrev.PUID = imanrel.RPRIMARY_OBJECTU)
       INNER JOIN INFODBA.PFORM form ON (imanrel.RSECONDARY_OBJECTU = form.PUID)
       INNER JOIN INFODBA.PZIMMERMISDETAILS zim_mis ON (form.RDATA_FILEU = zim_mis.PUID)
       INNER JOIN INFODBA.P_TEST_DETAILS test_details ON (zim_mis.PUID = test_details.PUID)
       INNER JOIN INFODBA.P_PART_ASSOCIATION ppa ON (ppa.PUID = test_details.PUID)
       INNER JOIN INFODBA.PMISDATAOBJECT mis ON (mis.PUID = ppa.PVALU_0)
       INNER JOIN INFODBA.PZIMTESTDETAILS zim_test_details ON (test_details.PVALU_0 = zim_test_details.PUID)
 WHERE Status.PNAME IN ('Current', 'BackLevel')

Business Rules

Complex 10-table JOIN through CMS schema (INFODBA)
Filters for Status 'Current' or 'BackLevel' only
Special timeout: Uses DataAccessOptions.MisDataTimeoutSeconds (default 60000 seconds)
Data transformation: ReleaseDate converted to local time
Uses CMS connection via CreateCmsConnectionAsync()

Scenario: Fetch MIS data from CMS

WHEN CMS connection is available and GetMisDataAsync() is called
THEN 10-table JOIN returns Current and BackLevel MIS records as async stream

Scenario: Handle long-running query

WHEN MIS data query may take extended time and query executes
THEN configured timeout allows completion without error

SQL Server Cache Operations

Requirement: LotFinderDB.GetUserSearches component

The system SHALL retrieve searches for a specific user.

Method Signature

Task<List<Search>> GetUserSearchesAsync(string userName, CancellationToken ct = default)

Query

SELECT s.ID, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT
FROM   dbo.Search s
WHERE  s.UserName = @userName
ORDER BY s.SubmitDT

Business Rules

Throws QueryException on error (does not return empty list on error)
Does not include Criteria or Results (lightweight query)
Ordered by SubmitDT ascending

Scenario: Get user's search history

WHEN user "jdoe" has 5 previous searches and GetUserSearchesAsync("jdoe") is called
THEN 5 Search objects are returned without Criteria/Results (lightweight)

Requirement: LotFinderDB.GetQueuedSearches component

The system SHALL retrieve all searches pending processing.

Method Signature

Task<List<Search>> GetQueuedSearchesAsync(CancellationToken ct = default)

Query

SELECT s.ID, s.UserName, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT
FROM   dbo.Search s
WHERE  s.Status < 3
ORDER BY s.SubmitDT

Business Rules

Status < 3 = New (0), Submitted (1), Started (2)
Throws QueryException on error
Ordered by SubmitDT ascending (FIFO processing)

Scenario: Worker polls for pending searches

WHEN 3 searches with Status = Submitted exist and GetQueuedSearchesAsync() is called
THEN 3 Search objects are returned ordered by SubmitDT (FIFO)

Scenario: Exclude completed searches

WHEN 2 searches with Status = Ended (3) exist and GetQueuedSearchesAsync() is called
THEN those searches are not included (Status < 3 filter)

Requirement: LotFinderDB.GetSearch component

The system SHALL retrieve a single search by ID with criteria.

Method Signature

Task<Search?> GetSearchAsync(int id, CancellationToken ct = default)

Query

SELECT s.UserName, s.Name, s.Status, s.SubmitDT, s.StartDT, s.EndDT,
       s.Criteria as CriteriaJSON
FROM   dbo.Search s
WHERE  s.ID = @id

Business Rules

Returns null if not found
Throws QueryException on database error
Deserializes CriteriaJSON to SearchCriteria object
Sets ID property manually after query (not in SELECT)

Requirement: LotFinderDB.GetSearchResults component

The system SHALL retrieve Excel results for a completed search.

Method Signature

Task<byte[]?> GetSearchResultsAsync(int id, CancellationToken ct = default)

Query

SELECT s.Results
FROM   dbo.Search AS s
WHERE  s.ID = @id

Business Rules

Returns null if not found
Throws QueryException on database error
Results is VARBINARY(MAX) containing Excel file

Requirement: LotFinderDB.SubmitSearch component

The system SHALL create a new search request.

Method Signature

Task<int> SubmitSearchAsync(Search search, CancellationToken ct = default)

Stored Procedure

EXEC dbo.usp_SubmitSearch
    @p_UserName = ...,
    @p_Name = ...,
    @p_Criteria = ...,
    @o_SearchID = ... OUTPUT

Business Rules

Sets Status = Submitted and SubmitDT = DateTime.UtcNow before insert
Serializes Criteria to JSON via ToJSON() method
Throws QueryException on error
Uses stored procedure with OUTPUT parameter for new ID

Requirement: LotFinderDB.SearchItems component

The system SHALL provide autocomplete search for items by number or description.

Method Signature

Task<List<Item>> SearchItemsAsync(string filter, CancellationToken ct = default)

Query

SELECT TOP 25 i.ShortItemNumber, i.ItemNumber, i.Description, i.LastUpdateDT
FROM   dbo.Item AS i
WHERE  i.ItemNumber LIKE '%' + @filter + '%' OR
       i.Description LIKE '%' + @filter + '%'
ORDER BY i.ItemNumber

Business Rules

Returns top 25 matches
Case-insensitive LIKE search on both ItemNumber and Description
Throws QueryException on error

Requirement: LotFinderDB.LookupItems component

The system SHALL provide batch lookup of items by exact item numbers.

Method Signature

Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default)

Query

SELECT i.ShortItemNumber, i.ItemNumber, i.Description, i.LastUpdateDT
FROM   dbo.Item AS i
INNER JOIN @itemNumbers AS i2 ON (i.ItemNumber = i2.ItemNumber)
ORDER BY i.ItemNumber

Business Rules

Uses table-valued parameter ItemNumberFilterParameter
Throws QueryException on error

Requirement: LotFinderDB.LookupWorkorders component

The system SHALL provide batch lookup of work orders by work order numbers.

Method Signature

Task<List<WorkOrder>> LookupWorkordersAsync(List<long> workorderNumbers, CancellationToken ct = default)

Query

SELECT *
FROM   dbo.WorkOrder AS wo
INNER JOIN @workOrderNumbers wo2 ON (wo.WorkOrderNumber = wo2.WorkOrderNumber)

Business Rules

Uses table-valued parameter WorkOrderFilterParameter
SELECT * returns all columns from WorkOrder view
Throws QueryException on error

Requirement: LotFinderDB.SearchWorkCenters / LookupWorkCenters component

The system SHALL provide autocomplete and batch lookup for work centers.

SearchWorkCenters Query

SELECT TOP 25 wc.Code, wc.Description, wc.LastUpdateDT
FROM   dbo.WorkCenter AS wc
WHERE  wc.Code LIKE '%' + @filter + '%' OR
       wc.Description LIKE '%' + @filter + '%'
ORDER BY wc.Code

LookupWorkCenters Query

SELECT wc.Code, wc.Description, wc.LastUpdateDT
FROM   dbo.WorkCenter AS wc
INNER JOIN @workCenterCodes wc2 ON (wc.Code = wc2.Code)
ORDER BY wc.Code

Requirement: LotFinderDB.SearchProfitCenters / LookupProfitCenters component

The system SHALL provide autocomplete and batch lookup for profit centers.

SearchProfitCenters Query

SELECT TOP 25 pc.Code, pc.Description, pc.LastUpdateDT
FROM   dbo.ProfitCenter AS pc
WHERE  pc.Code LIKE '%' + @filter + '%' OR
       pc.Description LIKE '%' + @filter + '%'
ORDER BY pc.Code

LookupProfitCenters Query

SELECT pc.Code, pc.Description, pc.LastUpdateDT
FROM   dbo.ProfitCenter AS pc
INNER JOIN @profitCenterCodes AS pc2 ON (pc.Code = pc2.Code)
ORDER BY pc.Code

Requirement: LotFinderDB.SearchUsers / LookupUsers component

The system SHALL provide autocomplete and batch lookup for JDE users.

SearchUsers Query

SELECT TOP 25 u.AddressNumber, COALESCE(u.UserID,' ') AS UserID, u.FullName, u.LastUpdateDT
FROM   dbo.JdeUser AS u
WHERE  u.UserID LIKE '%' + @filter + '%' OR
       u.FullName LIKE '%' + @filter + '%' OR
       CAST(u.AddressNumber AS VARCHAR(10)) LIKE '%' + @filter + '%'
ORDER BY u.UserID, u.FullName

LookupUsers Query

SELECT u.AddressNumber, u.UserID, u.FullName, u.LastUpdateDT
FROM   dbo.JdeUser AS u
INNER JOIN @userIDs u2 ON (u.UserID = u2.UserName OR CAST(u.AddressNumber AS VARCHAR(20)) = u2.UserName)
ORDER BY u.UserID

Business Rules

SearchUsers: COALESCE UserID to space if null
LookupUsers: Matches by UserID OR AddressNumber (cast to string)
Uses table-valued parameter OperatorFilterParameter

Requirement: LotFinderDB.LookupLots component

The system SHALL provide batch lookup of lots by lot number and item number.

Method Signature

Task<List<Lot>> LookupLotsAsync(List<LotViewModel> lots, CancellationToken ct = default)

Query

SELECT DISTINCT l.LotNumber, l.BranchCode, l.ShortItemNumber, l.ItemNumber,
       l.SupplierCode, l.LastUpdateDT
FROM   dbo.Lot AS l
INNER JOIN @lotNumbers ln ON (l.LotNumber = ln.ComponentLotNumber AND
       ((l.ItemNumber IS NULL AND ln.ItemNumber IS NULL) OR l.ItemNumber = ln.ItemNumber))

Business Rules

Uses table-valued parameter ComponentLotFilterParameter
Matches on LotNumber AND ItemNumber (with NULL handling)
DISTINCT to eliminate duplicates

Requirement: LotFinderDB.PostProcessMisData component

The system SHALL post-process imported MIS data to set obsolete dates.

Method Signature

Task PostProcessMisDataAsync(CancellationToken ct = default)

Query

SET ANSI_WARNINGS OFF;

-- Set ObsoleteDate for Current status when BackLevel exists
WITH cte AS (
  SELECT md.MisNumber, md.RevID, md.Status, MIN(md.ReleaseDate) Released
  FROM dbo.MisData AS md
  GROUP BY md.MisNumber, md.RevID, md.Status
)
UPDATE dbo.MisData
SET    ObsoleteDate = bl.Released
FROM   cte bl
WHERE  MisData.MisNumber = bl.MisNumber AND
       MisData.RevID = bl.RevID AND
       MisData.Status = 'Current' AND
       bl.Status = 'BackLevel';

-- Set ObsoleteDate for remaining records based on next revision
WITH cte AS (
  SELECT md.MisNumber, md.RevID, md.Status, MIN(md.ReleaseDate) Released
  FROM dbo.MisData AS md
  GROUP BY md.MisNumber, md.RevID, md.Status
)
UPDATE dbo.MisData
SET    ObsoleteDate = (SELECT TOP 1 nl.Released
                       FROM   cte nl
                       WHERE  MisData.MisNumber = nl.MisNumber AND
                              MisData.RevID < nl.RevID AND
                              MisData.Status = nl.Status
                       ORDER BY nl.RevID)
WHERE ObsoleteDate IS NULL;

ALTER INDEX [PK_MisData] ON [dbo].[MisData] REBUILD;

Business Rules

Two-phase update: first BackLevel, then next revision
Rebuilds primary key index after updates
Disables ANSI_WARNINGS for aggregate operations

Requirement: LotFinderDB.GetLastDataUpdates component

The system SHALL retrieve most recent successful update per table/type.

Method Signature

Task<List<DataUpdate>> GetLastDataUpdatesAsync(CancellationToken ct = default)

Query

WITH DU_CTE AS (
  SELECT du.*,
         ROW_NUMBER() OVER (PARTITION BY du.TableName, du.UpdateType ORDER BY du.StartDT DESC) RN
  FROM   dbo.DataUpdate AS du
)
SELECT cte.SourceSystem, cte.SourceData, cte.TableName, cte.StartDT, cte.EndDT,
       cte.UpdateType, cte.WasSuccessful, cte.NumberRecords
FROM   DU_CTE cte
WHERE  cte.RN = 1

Business Rules

Returns latest record per (TableName, UpdateType) combination
Used to determine incremental update windows

Requirement: LotFinderDB.GetTableSpec component

The system SHALL retrieve table schema for dynamic operations.

Method Signature

Task<TableSpec> GetTableSpecAsync(string tableName, CancellationToken ct = default)

Queries

-- Get columns
SELECT c.name AS Name,
       CASE t2.name
         WHEN 'varchar' THEN 'VARCHAR(' + CAST(c.max_length AS VARCHAR(10)) + ')'
         WHEN 'decimal' THEN 'DECIMAL(' + CAST(c.precision AS VARCHAR(4)) + ',' + CAST(c.scale AS VARCHAR(4)) + ')'
         ELSE UPPER(t2.name)
       END AS Definition
FROM   sys.columns c
INNER JOIN sys.types AS t2 ON (c.system_type_id = t2.system_type_id)
INNER JOIN sys.tables t ON (c.object_id = t.object_id)
WHERE  t.name = @name
ORDER BY c.column_id

-- Get primary key columns
SELECT COLUMN_NAME AS Name
FROM   INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE  OBJECTPROPERTY(OBJECT_ID(CONSTRAINT_SCHEMA + '.' + QUOTENAME(CONSTRAINT_NAME)), 'IsPrimaryKey') = 1 AND
       TABLE_NAME = @name
ORDER BY ORDINAL_POSITION

Business Rules

Used for dynamic temp table creation during bulk operations
Maps SQL Server types to DDL definitions

Requirement: LotFinderDB.RebuildIndices component

The system SHALL rebuild all indices on a table with SQL injection protection.

Method Signature

Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)

Query

ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95);

Valid Table Names (Whitelist)

The following table names are valid for index rebuilding:

Branch
DataUpdate
FunctionCode
Item
JdeUser
Lot
LotLocation
LotUsage_Curr
LotUsage_Hist
MisData
OrgHierarchy
ProfitCenter
RouteMaster
Search
StatusCode
WorkCenter
WorkOrder_Curr
WorkOrder_Hist
WorkOrderComponent_Curr
WorkOrderComponent_Hist
WorkOrderRouting
WorkOrderStep_Curr
WorkOrderStep_Hist
WorkOrderTime_Curr
WorkOrderTime_Hist

Business Rules

Table name MUST be validated against whitelist before execution
Throws ArgumentException if table name is not in whitelist
FILLFACTOR = 95 leaves 5% free space for inserts
Uses DataAccessOptions.RebuildIndexTimeoutSeconds for timeout

Scenario: Valid table name

WHEN RebuildIndicesAsync("WorkOrder_Curr") is called
THEN index rebuild executes successfully because table name is in whitelist

Scenario: Invalid table name (SQL injection attempt)

WHEN RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--") is called
THEN ArgumentException is thrown with message "Invalid table name"

Requirement: LotFinderDB.GenerateTableParameter component

The system SHALL create DataTable for table-valued parameters.

Method Signature

DataTable GenerateTableParameter<T>(List<T> keys)

Business Rules

Creates single-column DataTable with column named "Key"
Column type matches generic type T
Used for simple single-value TVPs

Domain Model Cross-Reference

Repository Method	Domain Model	Cache Table
JDE.GetWorkOrders	WorkOrder	WorkOrder_Curr/WorkOrder_Hist
JDE.GetWorkOrderSteps	WorkOrderStep	WorkOrderStep_Curr/WorkOrderStep_Hist
JDE.GetWorkOrderTimes	WorkOrderTime	WorkOrderTime_Curr/WorkOrderTime_Hist
JDE.GetWorkOrderRoutings	WorkOrderRouting	WorkOrderRouting
JDE.GetWorkOrderComponents	WorkOrderComponent	WorkOrderComponent_Curr/WorkOrderComponent_Hist
JDE.GetLots	Lot	Lot
JDE.GetLotUsages	LotUsage	LotUsage_Curr/LotUsage_Hist
JDE.GetLotLocations	LotLocation	LotLocation
JDE.GetItems	Item	Item
JDE.GetUsers	JdeUser	JdeUser
JDE.GetBranches	Branch	Branch
JDE.GetProfitCenters	ProfitCenter	ProfitCenter
JDE.GetWorkCenters	WorkCenter	WorkCenter
JDE.GetStatusCodes	StatusCode	StatusCode
JDE.GetFunctionCodes	FunctionCode	FunctionCode
JDE.GetOrgHierarchy	OrgHierarchy	OrgHierarchy
JDE.GetRouteMasters	RouteMaster	RouteMaster
CMS.GetMisData	MisData	MisData
LotFinderDB.GetSearch	Search	Search
LotFinderDB.GetLastDataUpdates	DataUpdate	DataUpdate

JDE Table Reference

JDE Table	Schema	Description
F4801	PRODDTA / ARCDTAPD	Work Order Master
F3112	PRODDTA / ARCDTAPD	Work Order Routing (Steps)
F31122	PRODDTA / ARCDTAPD	Work Order Time Transactions
F3112Z1	PRODDTA	Work Order Routing Transactions
F3111	PRODDTA / ARCDTAPD	Work Order Parts List (Components)
F4108	PRODDTA	Lot Master
F4111	PRODDTA / ARCDTAPD	Item Ledger (Cardex)
F41021	JDESTAGE	Item Location
F4101	PRODDTA	Item Master
F0101	PRODDTA	Address Book Master
F0092	SY920	User Profile
F0006	PRODDTA	Business Unit Master
F0005	JDESTAGE	User Defined Codes
F00192	PRODDTA	Media Object (Function Codes)
F30006	PRODDTA	Work Center Master
F3003	PRODDTA	Routing Master

Migration Notes

Legacy Pattern	New Pattern	Rationale
`System.Data.SqlClient`	`Microsoft.Data.SqlClient`	Modern SQL Server client with better performance and security
`Oracle.ManagedDataAccess.Client` + `DDTek.Oracle`	`Oracle.ManagedDataAccess.Core`	Consolidated to single Oracle driver for .NET Core/.NET 5+
Static partial classes	Instance-based repositories with interfaces	Testability, dependency injection, loose coupling
External .sql files	Embedded resources or `const string` fields	Deployment simplicity, no runtime file I/O
`EncryptionHelper.Decrypt` with hardcoded key	Azure Key Vault or .NET Secret Manager	Industry standard secrets management
`buffered: false` Dapper	`QueryUnbufferedAsync` with `IAsyncEnumerable<T>`	Native async streaming, memory efficiency
`string.Format` SQL (RebuildIndices)	Table name whitelist validation	SQL injection prevention
Synchronous methods	Async/await with `CancellationToken`	Scalability, responsiveness, cancellation support
NLog static logger	`ILogger<T>` injected + `BeginScope()` for context	Modern structured logging with DI
Exception wrapping with generic message	Custom typed exceptions (`DataAccessException`, `ConnectionException`, `QueryException`, `DataAccessTimeoutException`)	Better error handling, debugging, and error classification
`Config.*CS` static properties	`IConfiguration["ConnectionStrings:*"]`	Configuration abstraction, environment support
Hardcoded timeout constants	`IOptions<DataAccessOptions>` with configurable values	Centralized configuration, runtime flexibility
DataTable for TVPs	Keep DataTable (Dapper compatible)	Dapper TVP support, no breaking change needed
Return null/empty on error	Throw typed exceptions	Consistent error handling, explicit failure semantics

Resolved Design Decisions

The following questions from the legacy analysis have been resolved with architectural decisions:

Oracle driver consolidation: Consolidated to Oracle.ManagedDataAccess.Core for all Oracle connections (JDE, JDE Stage, CMS). Single driver simplifies deployment and maintenance.
External SQL files vs embedded resources: SQL queries stored as embedded resources or compile-time constants. Eliminates runtime file I/O and simplifies deployment.
Password encryption: Replaced custom EncryptionHelper.Decrypt with Azure Key Vault or .NET Secret Manager. Industry-standard approach with proper key management.
Streaming vs buffered: All large dataset queries use IAsyncEnumerable<T> with QueryUnbufferedAsync. Streaming pattern preserved for memory efficiency.
SQL injection in RebuildIndices: Implemented table name whitelist validation. Only explicitly listed table names are allowed.
Error handling inconsistency: Standardized on throwing typed exceptions. No methods return null/empty on error conditions.
GetUsers incremental bug: Documented as intentional behavior (full sync always for users). No incremental filtering parameter.
Special timeouts: Made configurable via DataAccessOptions. LotUsage and MisData have separate configurable timeout values.
Schema placeholders: Schema names (ProductionSchema, ArchiveSchema, StageSchema) are configurable in DataAccessOptions for environment-specific deployments.
StatusCode/LotLocation connection: These queries use CreateJdeStageConnectionAsync() for JDESTAGE schema views. Separate connection configuration allows different credentials if needed.
CMS connection visibility: All connection methods are on IDbConnectionFactory interface, providing consistent public access pattern.
Async support: Full async support with Task<T> and IAsyncEnumerable<T> return types. All methods accept CancellationToken.
Connection pooling: Connection pooling is configured at the connection string level. IDbConnectionFactory opens connections; callers are responsible for disposal.
Transaction support: Transactions handled at the service layer when needed. Repository methods are single-unit operations.
F41021_VIEW vs F41021: JDESTAGE views provide flattened/optimized access. Original design preserved; uses JDE Stage connection.
Archive methods: Explicitly defined as separate interface methods (GetWorkOrdersArchiveAsync, etc.) for clarity.
_FILTERED query variants: Filtered queries use same method with optional lastUpdateDT parameter. Null = full sync, non-null = incremental.

Codex Review Findings

The following observations were noted during legacy code analysis:

Mixed driver usage: Legacy code uses both Oracle.ManagedDataAccess and DDTek.Oracle. New design consolidates to single driver.
Password decryption location: Legacy decrypts in Config.*CS properties. New design retrieves secrets from secure store at connection time.
LotFinderDB timeout comment: Legacy comment says "ms" but value is seconds. New design uses explicit TimeoutSeconds naming.
Query repository SetQuery unused: Legacy SetQuery method writes to disk but appears unused. Not carried forward to new design.
Inconsistent error patterns: Some legacy methods return empty/null on error, others throw. New design standardizes on exceptions.

73 KiB Raw Blame History

Data Access Specification

Purpose

Source Reference

Requirements

Requirement: Repository Interface Pattern

Interface Definitions

Business Rules

Scenario: Repository injection

Requirement: Connection Factory Pattern

Interface Definition

Business Rules

Scenario: Create SQL Server connection

Scenario: Create Oracle connection

Requirement: Service Registration

Method Signature

Business Rules

Scenario: Register data access services

Requirement: Configuration Options

Options Class

Business Rules

Requirement: Connection factory pattern

Implementation Pattern

Business Rules

Scenario: Successful connection creation

Scenario: Connection failure handling

Requirement: Async streaming pattern

Implementation Pattern

Business Rules

Scenario: Stream large work order dataset

Scenario: Cancel streaming operation

Requirement: Schema placeholder replacement

Implementation Pattern

Business Rules

Scenario: Query uses production schema

Requirement: Table name whitelist validation

Implementation Pattern

Business Rules

Scenario: Valid table name accepted

Scenario: SQL injection attempt blocked

Requirement: Exception logging with scope context

Implementation Pattern

Business Rules

Scenario: Query exception with full context

Requirement: Search criteria extraction via SQL functions

Implementation Pattern

Business Rules

Available Extraction Functions

Scenario: Build search query with extraction functions

Requirement: Table-valued parameter support for lookups

Business Rules

Scenario: Batch lookup with TVP

Requirement: Service registration extension method

Implementation Pattern

Business Rules

Scenario: Register all data access services

Exception Handling

Requirement: Custom Exception Types

Exception Hierarchy

Business Rules

Scenario: Connection failure logging

Scenario: Query timeout

Query Management

Requirement: QueryRepository component

Business Rules

Scenario: Load query from embedded resource

Scenario: Query not found

Connection Management

Requirement: Connection Management - LotFinderDB component

Method: CreateLotFinderConnectionAsync

Business Rules

Scenario: Successful connection

Scenario: Connection failure

Requirement: Connection Management - JDE component

Method: CreateJdeConnectionAsync

Business Rules

Scenario: Successful JDE connection

Requirement: Connection Management - JDE Stage component

Method: CreateJdeStageConnectionAsync

Business Rules

73 KiB

Raw Blame History