Joseph Doherty
824 lines
38 KiB
Markdown
824 lines
38 KiB
Markdown
# Search Processing Specification
|
|
|
|
## Purpose
|
|
|
|
The search processing subsystem enables users to query manufacturing data (work orders, lots, items, operators, and work centers) from a locally cached JDE/CMS database. It accepts multi-dimensional filter criteria, dynamically generates SQL queries using a builder pattern, executes searches against the SQL Server cache using Microsoft.Data.SqlClient, and aggregates results including downstream work order tracking and MIS (Manufacturing Information System) data extraction. This specification targets .NET 10 with modern C# language features and patterns.
|
|
|
|
## Source Reference
|
|
|
|
| Legacy Files | Purpose |
|
|
|--------------|---------|
|
|
| `OLD/DataModel/Models/SearchCriteria.cs` | Core search criteria model storing filter parameters |
|
|
| `OLD/DataModel/ViewModels/SearchCriteriaViewModel.cs` | View model with enriched filter data for UI binding |
|
|
| `OLD/WorkerService/Models/Reporting/SearchModel.cs` | Reporting model combining criteria with filter entries and results |
|
|
| `OLD/WorkerService/Templates/QueryTemplate.cs` | T4-generated SQL query builder (to be replaced with builder pattern) |
|
|
| `OLD/WorkerService/Templates/QueryTemplateExtension.cs` | Partial class extension for QueryTemplate |
|
|
| `OLD/WorkerService/Helpers/SearchModelHelpers.cs` | Helper methods for model conversion and parameter creation |
|
|
| `OLD/WorkerService/Models/Reporting/*.cs` | Filter entry models and result models |
|
|
|
|
## Requirements
|
|
|
|
### Requirement: Search Criteria Model
|
|
|
|
The system SHALL support a search criteria model with eight distinct filter types that can be combined in any combination.
|
|
|
|
#### Inputs
|
|
|
|
- **Timespan Filter**: Optional minimum and maximum datetime values
|
|
- **Work Order Filter**: Collection of work order numbers (long integers)
|
|
- **Item Number Filter**: Collection of item numbers (strings)
|
|
- **Profit Center Filter**: Collection of profit center codes (strings)
|
|
- **Work Center Filter**: Collection of work center codes (strings)
|
|
- **Operator Filter**: Collection of operator user IDs (strings)
|
|
- **Component Lot Filter**: Collection of lot number/item number pairs
|
|
- **Item Operation MIS Filter**: Collection of item/operation/MIS number/MIS revision combinations
|
|
- **Extract MIS Data Flag**: Boolean indicating whether to include MIS data in results
|
|
|
|
#### Outputs
|
|
|
|
- Serializable `SearchCriteria` object stored as JSON in the `Search.CriteriaJSON` column
|
|
- `SearchModel` populated with enriched filter entries (descriptions, full names, etc.)
|
|
|
|
#### Business Rules
|
|
|
|
- All filter collections SHALL be initialized as empty lists by default
|
|
- Filter lists MAY contain zero or more entries
|
|
- The timespan filter is considered enabled when either `MinimumDT` or `MaximumDT` has a value
|
|
- Each filter type is considered enabled when its collection contains at least one entry
|
|
- Component lot filters MUST include both lot number and item number for proper matching
|
|
- Item/Operation/MIS filters MUST include all four fields: item number, operation number, MIS number, and MIS revision
|
|
|
|
#### Scenario: Create search with timespan filter only
|
|
|
|
- **WHEN** a user specifies minimum date of 2024-01-01 and maximum date of 2024-12-31
|
|
- **THEN** the system creates a SearchCriteria with `MinimumDT` = 2024-01-01 and `MaximumDT` = 2024-12-31
|
|
- **AND** `TimespanFilterEnabled` returns true
|
|
- **AND** all other filter collections remain empty
|
|
|
|
#### Scenario: Create search with multiple filter types
|
|
|
|
- **WHEN** a user specifies work order numbers [12345, 67890] and item numbers ["PART-001", "PART-002"]
|
|
- **THEN** the system creates a SearchCriteria with both collections populated
|
|
- **AND** `WorkOrderFilterEnabled` and `ItemNumberFilterEnabled` both return true
|
|
- **AND** other filter collections remain empty
|
|
|
|
#### Scenario: Create search with component lot filter
|
|
|
|
- **WHEN** a user specifies component lot "LOT123" for item "ITEM-ABC"
|
|
- **THEN** the system creates a ComponentLotFilter entry with both LotNumber and ItemNumber
|
|
- **AND** the filter is used to find work orders that consumed material from that lot
|
|
|
|
#### Scenario: Serialize and deserialize search criteria
|
|
|
|
- **WHEN** a SearchCriteria object is serialized to JSON and stored
|
|
- **THEN** the system can deserialize it back to the original object structure
|
|
- **AND** all filter collections and values are preserved
|
|
|
|
---
|
|
|
|
### Requirement: Query Builder Generation
|
|
|
|
The system SHALL dynamically generate SQL queries based on enabled filters using SqlKata fluent query builder (replacing the legacy T4 text template).
|
|
|
|
#### Migration Note: T4 Template Replacement with SqlKata
|
|
|
|
The legacy system uses a T4 text template (`QueryTemplate.tt`) for SQL generation. T4 templates have limited support in modern .NET SDK-style projects. The new implementation SHALL use **SqlKata** - a fluent SQL query builder that generates parameterized SQL and integrates well with Dapper.
|
|
|
|
**NuGet Package:** `SqlKata` and `SqlKata.Execution`
|
|
|
|
**Primary Query Building with SqlKata and Extraction Functions:**
|
|
|
|
```csharp
|
|
using SqlKata;
|
|
using SqlKata.Compilers;
|
|
|
|
public interface ISearchQueryBuilder
|
|
{
|
|
SearchQueryResult BuildSearchQuery(int searchId);
|
|
SearchQueryResult BuildMisQuery(int searchId);
|
|
SearchQueryResult BuildMisNonMatchQuery(int searchId);
|
|
}
|
|
|
|
public record SearchQueryResult(string Sql, IDictionary<string, object> Parameters);
|
|
|
|
public sealed class SqlKataSearchQueryBuilder : ISearchQueryBuilder
|
|
{
|
|
private readonly SqlServerCompiler _compiler = new();
|
|
|
|
public SearchQueryResult BuildSearchQuery(int searchId)
|
|
{
|
|
// Query uses SQL extraction functions to get criteria values
|
|
// No need to pass filter collections from C# - just the SearchId
|
|
|
|
// Generated SQL populates temp tables from extraction functions:
|
|
// INSERT INTO #P_WorkOrders SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(@SearchId)
|
|
// INSERT INTO #P_ItemNumbers SELECT ItemNumber FROM dbo.fn_GetSearchItemNumbers(@SearchId)
|
|
// etc.
|
|
|
|
var query = new Query("WorkOrder_Curr as wo")
|
|
.Select("wo.*", "wo.WorkOrderNumber", "wo.ItemNumber", "wo.StatusCode");
|
|
|
|
// Conditional joins based on temp table population (determined at runtime)
|
|
// Query builder generates SQL that checks IF EXISTS on temp tables
|
|
|
|
var compiled = _compiler.Compile(query);
|
|
return new SearchQueryResult(compiled.Sql, new Dictionary<string, object> { ["SearchId"] = searchId });
|
|
}
|
|
}
|
|
```
|
|
|
|
**Extraction Function Integration:**
|
|
|
|
The query builder generates SQL that populates temporary filter tables from extraction functions:
|
|
|
|
```sql
|
|
-- Generated SQL fragment example:
|
|
-- Populate temp tables from extraction functions
|
|
INSERT INTO #P_WorkOrders SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(@SearchId);
|
|
INSERT INTO #P_ItemNumbers SELECT ItemNumber FROM dbo.fn_GetSearchItemNumbers(@SearchId);
|
|
INSERT INTO #P_ProfitCenters SELECT Code FROM dbo.fn_GetSearchProfitCenters(@SearchId);
|
|
INSERT INTO #P_WorkCenters SELECT Code FROM dbo.fn_GetSearchWorkCenters(@SearchId);
|
|
INSERT INTO #P_OperatorIDs SELECT OperatorID FROM dbo.fn_GetSearchOperatorIDs(@SearchId);
|
|
INSERT INTO #P_ComponentLots SELECT LotNumber, ItemNumber FROM dbo.fn_GetSearchComponentLots(@SearchId);
|
|
INSERT INTO #P_PartOperations SELECT ItemNumber, OperationNumber, MisNumber, MisRevision FROM dbo.fn_GetSearchPartOperations(@SearchId);
|
|
|
|
-- Check which filters are active based on temp table population
|
|
DECLARE @HasWorkOrderFilter BIT = CASE WHEN EXISTS(SELECT 1 FROM #P_WorkOrders) THEN 1 ELSE 0 END;
|
|
-- etc.
|
|
```
|
|
|
|
This approach eliminates the need to pass filter collections from C# to SQL, simplifying the interface to a single `searchId` parameter.
|
|
|
|
**Work Order Traversal (Stored Procedure):**
|
|
|
|
The iterative downstream work order traversal is implemented as a stored procedure (`dbo.TraverseWorkOrders`) due to its iterative nature with temp tables and MERGE operations. The maximum iteration count is configurable via `SearchProcessingOptions.MaxTraversalIterations` (default: 20):
|
|
|
|
```csharp
|
|
public class SearchProcessingOptions
|
|
{
|
|
public const string SectionName = "SearchProcessing";
|
|
public int MaxTraversalIterations { get; set; } = 20;
|
|
}
|
|
```
|
|
|
|
**TraverseWorkOrders Stored Procedure Interface:**
|
|
|
|
```csharp
|
|
public interface IWorkOrderTraversalService
|
|
{
|
|
Task<List<long>> TraverseDownstreamAsync(
|
|
IEnumerable<long> seedWorkOrders,
|
|
CancellationToken ct = default);
|
|
}
|
|
|
|
public sealed class WorkOrderTraversalService : IWorkOrderTraversalService
|
|
{
|
|
private readonly IDbConnectionFactory _connectionFactory;
|
|
private readonly IOptions<SearchProcessingOptions> _options;
|
|
|
|
public async Task<List<long>> TraverseDownstreamAsync(
|
|
IEnumerable<long> seedWorkOrders,
|
|
CancellationToken ct = default)
|
|
{
|
|
await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
|
|
|
|
var maxIterations = _options.Value.MaxTraversalIterations;
|
|
|
|
return (await connection.QueryAsync<long>(
|
|
"EXEC dbo.TraverseWorkOrders @p_SeedWorkOrders, @p_MaxIterations",
|
|
new {
|
|
p_SeedWorkOrders = seedWorkOrders.ToList().AsTableValuedParameter("dbo.WorkOrderList"),
|
|
p_MaxIterations = maxIterations
|
|
})).AsList();
|
|
}
|
|
}
|
|
```
|
|
|
|
**MIS Query Builder (Separate):**
|
|
|
|
For MIS data extraction, a separate `IMisQueryBuilder` SHALL be used to generate MIS-specific queries:
|
|
|
|
```csharp
|
|
public interface IMisQueryBuilder
|
|
{
|
|
SearchQueryResult BuildMisDataQuery(SearchModel model);
|
|
SearchQueryResult BuildMisNonMatchQuery(SearchModel model);
|
|
}
|
|
|
|
public sealed class SqlKataMisQueryBuilder : IMisQueryBuilder
|
|
{
|
|
private readonly SqlServerCompiler _compiler = new();
|
|
|
|
public SearchQueryResult BuildMisDataQuery(SearchModel model)
|
|
{
|
|
var query = new Query("MisData as m")
|
|
.Select("m.*")
|
|
.Join("WorkOrderStep_Curr as wos", q => q
|
|
.On("wos.ItemNumber", "m.ItemNumber")
|
|
.On("wos.OperationNumber", "m.OperationNumber"));
|
|
// Additional MIS-specific filtering...
|
|
var compiled = _compiler.Compile(query);
|
|
return new SearchQueryResult(compiled.Sql, compiled.NamedBindings);
|
|
}
|
|
|
|
public SearchQueryResult BuildMisNonMatchQuery(SearchModel model)
|
|
{
|
|
// Investigation query for router mismatches
|
|
var query = new Query("WorkOrderStep_Curr as wos")
|
|
.LeftJoin("MisData as m", q => q
|
|
.On("wos.ItemNumber", "m.ItemNumber")
|
|
.On("wos.OperationNumber", "m.OperationNumber"))
|
|
.WhereNull("m.ItemNumber"); // Non-matches
|
|
var compiled = _compiler.Compile(query);
|
|
return new SearchQueryResult(compiled.Sql, compiled.NamedBindings);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Streaming vs Materialized Results:**
|
|
|
|
The system SHALL provide two methods for result retrieval:
|
|
|
|
```csharp
|
|
public interface ISearchResultReader
|
|
{
|
|
/// <summary>Streaming - returns results as they are read (memory-efficient for large result sets)</summary>
|
|
IAsyncEnumerable<SearchResult> StreamResultsAsync(int searchId, CancellationToken ct = default);
|
|
|
|
/// <summary>Materialized - loads all results into memory (simpler for small result sets)</summary>
|
|
Task<List<SearchResult>> GetResultsAsync(int searchId, CancellationToken ct = default);
|
|
}
|
|
```
|
|
|
|
**SqlKata Benefits:**
|
|
|
|
- **Parameterized by default** - SQL injection protection built-in; all parameters use `@p_*` naming convention for clarity
|
|
- **Fluent API** - Readable, composable query building
|
|
- **Testable** - Unit test query building without database
|
|
- **Type-safe** - Compile-time checking of method calls
|
|
- **SQL Server optimized** - SqlServerCompiler generates optimized T-SQL
|
|
|
|
**Parameter Naming Convention:**
|
|
|
|
All SQL parameters SHALL use the `@p_` prefix for consistency:
|
|
- `@p_SeedWorkOrders` (not `@SeedWorkOrders`)
|
|
- `@p_MaxIterations` (not `@MaxIterations`)
|
|
- `@p_MinimumDT`, `@p_MaximumDT`, etc.
|
|
|
|
#### Inputs
|
|
|
|
- `SearchModel` containing filter criteria and enabled filter flags
|
|
- Table-valued parameters for each enabled filter type
|
|
|
|
#### Outputs
|
|
|
|
- Complete T-SQL query string including:
|
|
- Temporary table creation statements
|
|
- Filter parameter population
|
|
- Work order flagging logic
|
|
- Downstream work order traversal
|
|
- Final result selection
|
|
- Optional MIS data extraction queries
|
|
|
|
#### Business Rules
|
|
|
|
- The query builder MUST create a `#Temp_WO` temporary table to track flagged work orders
|
|
- The builder SHALL include SQL segments only for enabled filters (conditional generation)
|
|
- Work order step searching is triggered when any of these filters are enabled: timespan, profit center, work center, or operator
|
|
- The query MUST perform iterative downstream traversal (up to 20 iterations) to find work orders that received material from flagged work orders
|
|
- Split orders (orders created by splitting a parent order) SHALL be automatically included
|
|
- Query parameters MUST be passed as table-valued parameters for multi-value filters
|
|
|
|
#### Scenario: Generate query with work order filter only
|
|
|
|
- **WHEN** search criteria includes only work order numbers [12345, 67890]
|
|
- **THEN** the system generates SQL that creates `#Temp_WO` with ManuallySpecified flag
|
|
- **AND** includes MERGE statements to add specified work orders
|
|
- **AND** includes downstream traversal logic for related work orders
|
|
- **AND** excludes timespan, profit center, work center, and operator filter logic
|
|
|
|
#### Scenario: Generate query with profit center filter
|
|
|
|
- **WHEN** search criteria includes profit center codes ["PC01", "PC02"]
|
|
- **THEN** the system generates SQL that creates `#P_WorkCenters` temp table
|
|
- **AND** populates it by joining profit center codes to work centers via `OrgHierarchy`
|
|
- **AND** includes work order step search logic with profit center join
|
|
- **AND** sets `ShouldSearchSteps()` to return true
|
|
|
|
#### Scenario: Generate query with all filters enabled
|
|
|
|
- **WHEN** search criteria includes values for all eight filter types
|
|
- **THEN** the system generates SQL with all conditional segments included
|
|
- **AND** joins all filter temp tables appropriately
|
|
- **AND** includes MIS extraction queries when `ExtractMisData` is true
|
|
|
|
#### Scenario: Generate query with operator filter
|
|
|
|
- **WHEN** search criteria includes operator user IDs ["USER1", "USER2"]
|
|
- **THEN** the system generates SQL that creates `#P_OperatorIDs` temp table
|
|
- **AND** resolves user IDs to address numbers via `JdeUser` table
|
|
- **AND** joins work order time records to filter by operator
|
|
|
|
---
|
|
|
|
### Requirement: Filter Entry Processing
|
|
|
|
The system SHALL enrich raw filter values with descriptive information from reference tables using C# record types for immutability.
|
|
|
|
#### Filter Entry Location
|
|
|
|
Filter entry types SHALL be located in the `JdeScoping.Core.Models` namespace within the Domain Models project, not in a separate processing project. This ensures proper separation of domain models from processing logic.
|
|
|
|
#### Migration Note: Record Types
|
|
|
|
The legacy filter entry classes should be converted to C# record types for value semantics and immutability:
|
|
|
|
```csharp
|
|
// File: JdeScoping.Core/Models/FilterEntries.cs
|
|
namespace JdeScoping.Core.Models;
|
|
|
|
// Legacy class pattern
|
|
public class WorkOrderFilterEntry
|
|
{
|
|
public long WorkOrderNumber { get; set; }
|
|
public string ItemNumber { get; set; }
|
|
}
|
|
|
|
// Modern record pattern
|
|
public record WorkOrderFilterEntry(long WorkOrderNumber, string ItemNumber);
|
|
|
|
// With attributes for Excel output configuration
|
|
[OutputTable(TabName = "Work Order Filter", ShowHeader = true)]
|
|
public record WorkOrderFilterEntry(
|
|
[property: OutputColumn(Order = 10, HeaderText = "Work Order Number")]
|
|
long WorkOrderNumber,
|
|
[property: OutputColumn(Order = 20, HeaderText = "Item Number")]
|
|
string ItemNumber
|
|
);
|
|
```
|
|
|
|
Records provide:
|
|
- Immutability by default (init-only properties)
|
|
- Value-based equality
|
|
- Built-in `ToString()` and deconstruction
|
|
- Cleaner syntax for data transfer objects
|
|
|
|
#### Inputs
|
|
|
|
- Raw filter values from `SearchCriteria` (IDs, codes, numbers)
|
|
- Reference data from `LotFinderDB` lookup methods
|
|
|
|
#### Outputs
|
|
|
|
- Enriched filter entry records containing:
|
|
- `WorkOrderFilterEntry`: WorkOrderNumber + ItemNumber
|
|
- `ItemNumberFilterEntry`: ItemNumber + ItemDescription
|
|
- `ProfitCenterFilterEntry`: Code + Description
|
|
- `WorkCenterFilterEntry`: Code + Description
|
|
- `OperatorFilterEntry`: AddressNumber + UserID + FullName
|
|
- `ComponentLotFilterEntry`: LotNumber + ItemNumber
|
|
- `ItemOperationMisFilterEntry`: ItemNumber + OperationNumber + MisNumber + MisRevision
|
|
|
|
#### Business Rules
|
|
|
|
- Filter entries SHALL be populated via `LotFinderDB` lookup methods
|
|
- Lookup methods MUST return empty collections for invalid or unmatched values
|
|
- Operator filter entries MUST resolve user IDs to address numbers for query execution
|
|
- Component lot entries MUST include both lot number and item number for proper traceability
|
|
|
|
#### Scenario: Enrich work order filter entries
|
|
|
|
- **WHEN** the system processes work order numbers [12345, 67890]
|
|
- **THEN** it calls `LotFinderDB.LookupWorkorders()` with the numbers
|
|
- **AND** creates `WorkOrderFilterEntry` records with work order number and item number
|
|
- **AND** includes only work orders that exist in the database
|
|
|
|
#### Scenario: Resolve operator user IDs to address numbers
|
|
|
|
- **WHEN** the system processes operator user IDs ["JSMITH", "MBROWN"]
|
|
- **THEN** it calls `LotFinderDB.LookupUsers()` to get JDE user records
|
|
- **AND** creates `OperatorFilterEntry` records with AddressNumber, UserID, and FullName
|
|
- **AND** the AddressNumber is used for joining to work order time records
|
|
|
|
#### Scenario: Handle invalid filter values
|
|
|
|
- **WHEN** the system processes item numbers containing non-existent items
|
|
- **THEN** the lookup returns only matching items
|
|
- **AND** non-existent item numbers are silently excluded from results
|
|
|
|
#### Scenario: Process component lot filter with item validation
|
|
|
|
- **WHEN** the system processes a component lot filter entry
|
|
- **THEN** it validates both lot number and item number exist together
|
|
- **AND** uses the combination to trace downstream work orders via `WorkOrderComponent` and `LotUsage` tables
|
|
|
|
---
|
|
|
|
### Requirement: Work Order Flagging and Traversal
|
|
|
|
The system SHALL flag work orders for inclusion based on multiple criteria and traverse downstream relationships.
|
|
|
|
#### Inputs
|
|
|
|
- Filter criteria from `SearchModel`
|
|
- Work order relationships from database tables:
|
|
- `WorkOrder` (parent/child via ParentWorkOrderNumber)
|
|
- `WorkOrderComponent` (parts list relationships)
|
|
- `LotUsage` (CARDEX material usage)
|
|
- `WorkOrderStep` (operation details)
|
|
- `WorkOrderTime` (operator time records)
|
|
|
|
#### Outputs
|
|
|
|
- `#Temp_WO` temporary table containing:
|
|
- `WorkOrderNumber`: Primary key
|
|
- `LotNumber`: Associated lot number
|
|
- `BranchCode`: Branch code for the work order
|
|
- `ShortItemNumber`: Item's short number
|
|
- `ManuallySpecified`: Flag indicating direct specification
|
|
- `SplitOrder`: Flag indicating split from flagged order
|
|
- `CARDEX`: Flag indicating material receipt from flagged order (F4111) - **Note: Also set for WorkOrderComponent matches**
|
|
- `PartsList`: Flag indicating parts list relationship (F3111)
|
|
- `Flagged`: Flag indicating filter criteria match
|
|
|
|
#### Business Rules
|
|
|
|
- Work orders directly specified in the filter SHALL be marked with `ManuallySpecified = 1`
|
|
- Work orders split from flagged orders (matching `ParentWorkOrderNumber`) SHALL be marked with `SplitOrder = 1`
|
|
- Work orders receiving material from flagged lots via `WorkOrderComponent` SHALL be marked with `CARDEX = 1` (not PartsList as might be expected)
|
|
- Work orders receiving material from flagged lots via `LotUsage` SHALL be marked with `CARDEX = 1`
|
|
- Work orders matching filter criteria (timespan, profit center, work center, operator) SHALL be marked with `Flagged = 1`
|
|
- Downstream traversal SHALL iterate up to 20 times to find all related work orders
|
|
- Traversal stops when no new work orders are found or maximum iterations reached
|
|
|
|
#### Scenario: Flag manually specified work orders
|
|
|
|
- **WHEN** work order 12345 is specified in the work order filter
|
|
- **THEN** the system marks it with `ManuallySpecified = 1` in `#Temp_WO`
|
|
- **AND** finds any split orders from 12345 and marks them with `SplitOrder = 1`
|
|
|
|
#### Scenario: Trace downstream work orders via parts list
|
|
|
|
- **WHEN** work order 12345 produces lot "LOT-A" for item "ITEM-001"
|
|
- **AND** work order 67890 consumes "LOT-A" for "ITEM-001" via parts list
|
|
- **THEN** work order 67890 is added to `#Temp_WO` with `CARDEX = 1`
|
|
|
|
#### Scenario: Trace downstream work orders via CARDEX
|
|
|
|
- **WHEN** work order 12345 produces lot "LOT-A"
|
|
- **AND** work order 67890 consumes "LOT-A" via LotUsage (CARDEX)
|
|
- **THEN** work order 67890 is added to `#Temp_WO` with `CARDEX = 1`
|
|
|
|
#### Scenario: Multi-level downstream traversal
|
|
|
|
- **WHEN** work order A produces material consumed by work order B
|
|
- **AND** work order B produces material consumed by work order C
|
|
- **THEN** the iterative traversal finds A, B, and C
|
|
- **AND** marks each with appropriate flags based on relationship type
|
|
|
|
---
|
|
|
|
### Requirement: Search Result Aggregation
|
|
|
|
The system SHALL aggregate flagged work orders into structured result objects with complete details.
|
|
|
|
#### Inputs
|
|
|
|
- Flagged work orders from `#Temp_WO`
|
|
- Work order details from `WorkOrder` table
|
|
- Item details from `Item` table
|
|
- Status details from `StatusCode` table
|
|
- Latest step from `WorkOrderStep` table
|
|
- Scrap totals from `WorkOrderTotalScrap` table
|
|
|
|
#### Outputs
|
|
|
|
- `SearchResult` objects containing:
|
|
- Work order identification (number, branch code, lot number, item number)
|
|
- Item details (planning family, stocking type)
|
|
- Quantities (order, held, scrapped, shipped)
|
|
- Latest operation step details
|
|
- Status information (code, description, update timestamp)
|
|
- Inclusion reason derived from flag combination
|
|
|
|
#### Business Rules
|
|
|
|
- Each result SHALL include the latest operation step (highest EndDT, then highest StepNumber)
|
|
- Inclusion reason SHALL be calculated with strict priority order:
|
|
1. **ManuallySpecified** (highest) - Work order was directly specified in filter
|
|
2. **Flagged** - Work order matched timespan/profit center/work center/operator criteria
|
|
3. **ComponentUsage** - Work order received material from a flagged work order
|
|
4. **SplitOrder** (lowest) - Work order was split from a flagged parent order
|
|
- Component usage reason SHALL distinguish between CARDEX only, Parts List only, or both
|
|
- Scrapped quantity SHALL default to 0 when no scrap records exist
|
|
- Results with null latest step are still included (work orders without operations)
|
|
- Inclusion reason MAY return "UNKNOWN" when no flags are set (edge case)
|
|
|
|
#### Scenario: Generate inclusion reason for manually specified order
|
|
|
|
- **WHEN** a work order is marked with `ManuallySpecified = 1`
|
|
- **THEN** the `InclusionReason` property returns "ManuallySpecified"
|
|
|
|
#### Scenario: Generate inclusion reason for flagged order
|
|
|
|
- **WHEN** a work order is marked with `Flagged = 1` and `ManuallySpecified = 0`
|
|
- **THEN** the `InclusionReason` property returns "Flagged"
|
|
|
|
#### Scenario: Generate inclusion reason for component usage with both sources
|
|
|
|
- **WHEN** a work order is marked with `CARDEX = 1` and `PartsList = 1`
|
|
- **THEN** the `InclusionReason` property returns "ComponentUsage (CARDEX + Parts List)"
|
|
|
|
#### Scenario: Retrieve latest operation step for work order
|
|
|
|
- **WHEN** work order 12345 has steps 10, 20, 30 with EndDT values
|
|
- **THEN** the result includes the step with the highest EndDT
|
|
- **AND** if EndDT values tie, the highest StepNumber wins
|
|
|
|
---
|
|
|
|
### Requirement: MIS Data Extraction
|
|
|
|
The system SHALL optionally extract Manufacturing Information System (MIS) data for matching work orders.
|
|
|
|
#### Inputs
|
|
|
|
- `ExtractMisData` flag from search criteria
|
|
- `ItemOperationMisFilter` entries (when filtering by specific MIS)
|
|
- Work order step data joined with MIS matching function `dbo.fn_MatchMIS`
|
|
|
|
#### Outputs
|
|
|
|
- `MisSearchResult` objects containing:
|
|
- Item and MIS identification
|
|
- Matching indicators (RoutingMatch, MasterMatch)
|
|
- Test descriptions, sampling information
|
|
- Tools/gauges and work instructions
|
|
- `MisNonMatchSearchResult` objects containing:
|
|
- Work orders with steps that did not match MIS records
|
|
- Added job step indicators
|
|
- Matched job step suggestions based on work center and function code
|
|
|
|
#### Business Rules
|
|
|
|
- MIS extraction is only performed when `ExtractMisData = true`
|
|
- The `fn_MatchMIS` table-valued function handles MIS record matching logic
|
|
- RoutingMatch indicates match to F3112Z1 (work order routing) records
|
|
- MasterMatch indicates match to F3003 (item master routing) records
|
|
- Non-match results include work orders where neither RoutingMatch nor MasterMatch is true, or MisNumber is null
|
|
- Added job step detection uses routing type and F3111 records to identify manually added steps
|
|
- Matched job step number suggests alternative step matching by work center and function code
|
|
- **Note**: MIS extraction does NOT join `#Temp_WO` - work order/component lot filters do NOT constrain MIS results
|
|
- **Note**: Timespan filtering in MIS extraction requires BOTH min and max values for the combined condition; separate handling exists for min-only or max-only cases
|
|
|
|
#### Scenario: Extract MIS data for filtered work orders
|
|
|
|
- **WHEN** search criteria has `ExtractMisData = true`
|
|
- **THEN** the system generates MIS extraction queries
|
|
- **AND** populates `#TempMisData` with MIS-matched step data
|
|
- **AND** returns `MisSearchResult` list with matching records
|
|
|
|
#### Scenario: Identify MIS non-matches for investigation
|
|
|
|
- **WHEN** MIS data is extracted and a step has no routing or master match
|
|
- **THEN** the system includes it in `MisNonMatchSearchResult` list
|
|
- **AND** indicates whether the job step was added (not in original routing)
|
|
- **AND** suggests matched job step number if one exists with same work center and function code
|
|
|
|
#### Scenario: Filter by specific MIS number and revision
|
|
|
|
- **WHEN** search criteria includes ItemOperationMisFilter with specific MIS entries
|
|
- **THEN** the query joins to `#P_PartOperations` temp table
|
|
- **AND** only returns work order steps matching the specified item/operation/MIS/revision combinations
|
|
- **AND** the WorkOrderTime UNION branch is skipped (only WorkOrderStep branch used)
|
|
|
|
#### Scenario: Handle NMR routing type
|
|
|
|
- **WHEN** a work order has routing type "NMR" (No Master Routing)
|
|
- **THEN** all job steps are considered added (`WasJobStepAdded = 1`)
|
|
- **AND** matched job step number is null (no routing to match against)
|
|
|
|
---
|
|
|
|
### Requirement: Simplified Query Parameter Passing
|
|
|
|
The system SHALL pass only the SearchId parameter to search queries, with filter extraction handled by SQL functions.
|
|
|
|
#### Migration Note: SqlClient Package Change (Required)
|
|
|
|
Replace `System.Data.SqlClient` with `Microsoft.Data.SqlClient`:
|
|
|
|
```csharp
|
|
// Legacy
|
|
using System.Data.SqlClient;
|
|
|
|
// Modern .NET 10
|
|
using Microsoft.Data.SqlClient;
|
|
```
|
|
|
|
#### Simplified Parameter Model
|
|
|
|
With extraction functions, the query builder interface is simplified:
|
|
|
|
```csharp
|
|
public interface ISearchQueryBuilder
|
|
{
|
|
SearchQueryResult BuildSearchQuery(int searchId);
|
|
SearchQueryResult BuildMisQuery(int searchId);
|
|
SearchQueryResult BuildMisNonMatchQuery(int searchId);
|
|
}
|
|
|
|
// Query execution only needs SearchId
|
|
var result = queryBuilder.BuildSearchQuery(searchId);
|
|
var results = await connection.QueryAsync<SearchResult>(result.Sql, new { SearchId = searchId });
|
|
```
|
|
|
|
#### Business Rules
|
|
|
|
- Search queries SHALL accept only `searchId` as the filter parameter
|
|
- SQL extraction functions handle JSON parsing and filter population
|
|
- TVP infrastructure for search execution has been removed
|
|
- TVPs remain in use for reference data lookups (LookupItemsAsync, etc.)
|
|
- All database operations SHOULD use async methods (`QueryAsync`, `ExecuteAsync`)
|
|
|
|
#### Scenario: Execute search with simplified parameter
|
|
|
|
- **WHEN** search ID 123 is processed
|
|
- **THEN** query builder generates SQL with `@SearchId` parameter only
|
|
- **AND** extraction functions populate temporary filter tables from Search.Criteria JSON
|
|
- **AND** no DataTable or TVP creation is needed for search execution
|
|
|
|
---
|
|
|
|
### Requirement: Output Column Configuration
|
|
|
|
The system SHALL use attribute-based configuration for Excel output column formatting.
|
|
|
|
#### Migration Note: Attributes and Reflection
|
|
|
|
The attribute-based pattern works well in .NET 10 with standard reflection. For high-performance scenarios, consider:
|
|
|
|
1. **Source generators** - Generate column metadata at compile time
|
|
2. **Cached reflection** - Cache `PropertyInfo` and attribute lookups on first access
|
|
|
|
```csharp
|
|
// Attribute pattern preserved from legacy (works with records)
|
|
[OutputTable(TabName = "Search Results", TableName = "Search_Results")]
|
|
public record SearchResult(
|
|
[property: OutputColumn(Order = 10, HeaderText = "Work Order")]
|
|
long WorkOrderNumber,
|
|
|
|
[property: OutputColumn(Order = 20, HeaderText = "Status Date",
|
|
Format = OutputColumnAttribute.DATE_FORMAT)]
|
|
DateTime? StatusUpdateDT
|
|
);
|
|
|
|
// Cached reflection helper
|
|
public static class OutputColumnCache
|
|
{
|
|
private static readonly ConcurrentDictionary<Type, IReadOnlyList<OutputColumn>> _cache = new();
|
|
|
|
public static IReadOnlyList<OutputColumn> GetColumns<T>() =>
|
|
_cache.GetOrAdd(typeof(T), type => BuildColumns(type));
|
|
}
|
|
```
|
|
|
|
#### Inputs
|
|
|
|
- `OutputColumnAttribute` on result model properties:
|
|
- `Order`: Display order (integer)
|
|
- `HeaderText`: Column header text
|
|
- `Format`: Excel format string
|
|
- `AutoWidth`: Whether to auto-size column
|
|
- `Width`: Manual width when AutoWidth is false
|
|
- `WrapText`: Whether to wrap text in cells
|
|
- `OutputTableAttribute` on result model classes:
|
|
- `TabName`: Excel worksheet tab name
|
|
- `TableName`: Table identifier for styling
|
|
- `ShowHeader`: Whether to display merged header
|
|
|
|
#### Outputs
|
|
|
|
- `OutputColumn` objects combining property info with attribute metadata
|
|
- Configured Excel worksheets with proper formatting
|
|
|
|
#### Business Rules
|
|
|
|
- Columns are ordered by the `Order` attribute value (ascending)
|
|
- Properties without `OutputColumnAttribute` are excluded from output
|
|
- Standard format `"@"` treats values as text
|
|
- Date format `"[$-409]MM/dd/yyyy;@"` applies US English date formatting
|
|
- Timestamp format `"[$-409]m/d/yy h:mm AM/PM;@"` includes time component
|
|
- Wrapped columns use fixed width of 65 characters by default
|
|
- Filter entry tables show headers; result tables do not
|
|
|
|
#### Scenario: Configure date column formatting
|
|
|
|
- **WHEN** a property has `Format = OutputColumnAttribute.DATE_FORMAT`
|
|
- **THEN** Excel displays values in MM/dd/yyyy format
|
|
- **AND** the column width auto-sizes to fit content
|
|
|
|
#### Scenario: Configure wrapped text column
|
|
|
|
- **WHEN** a property has `WrapText = true` and `AutoWidth = false`
|
|
- **THEN** Excel wraps long text within the cell
|
|
- **AND** column width is set to `WRAPPED_COLUMN_WIDTH` (65 characters)
|
|
|
|
#### Scenario: Order columns in output
|
|
|
|
- **WHEN** SearchResult has properties with Order values 10, 20, 30
|
|
- **THEN** Excel columns appear in ascending order by the Order value
|
|
- **AND** gaps in Order values are allowed
|
|
|
|
#### Scenario: Configure tab name for result type
|
|
|
|
- **WHEN** SearchResult has `OutputTableAttribute` with `TabName = "Search Results"`
|
|
- **THEN** the Excel worksheet tab is named "Search Results"
|
|
- **AND** the table is styled with identifier "Search_Results"
|
|
|
|
---
|
|
|
|
## Migration Notes
|
|
|
|
| Legacy Pattern | New Pattern | Rationale |
|
|
|----------------|-------------|-----------|
|
|
| T4 Text Template (`QueryTemplate.tt`) | **SqlKata** fluent query builder | SqlKata provides parameterized SQL by default, fluent API, composable queries, testability, and SQL Server-optimized output; T4 templates are not well-supported in modern .NET SDK projects |
|
|
| Iterative work order traversal in T4 | Stored procedure `dbo.TraverseWorkOrders` | Iterative logic with temp tables and MERGE is better handled server-side; stored procedure reduces round trips and is transactionally consistent |
|
|
| `System.Data.SqlClient` | `Microsoft.Data.SqlClient` | Legacy package is deprecated and no longer receives security updates; Microsoft.Data.SqlClient is the supported replacement |
|
|
| Filter entry classes | C# record types | Records provide immutability, value equality, and cleaner syntax for DTOs |
|
|
| Synchronous Dapper calls | Async Dapper methods (`QueryAsync`, `ExecuteAsync`) | Async operations improve scalability and are idiomatic in modern .NET |
|
|
| `DataTable.AsTableValuedParameter()` | DataTable (unchanged) or `IEnumerable<SqlDataRecord>` | DataTable approach works; SqlDataRecord is more efficient for large datasets |
|
|
| Conditional string concatenation in T4 | SqlKata fluent conditional methods (`.Join()`, `.WhereIn()`, `.When()`) | SqlKata handles conditional query building natively with compile-time safety |
|
|
| `SqlMapper.ICustomQueryParameter` (Dapper) | Dapper table-valued parameters (unchanged) | Dapper's API remains compatible in .NET 10 |
|
|
| Attribute-based output configuration | Attributes with optional cached reflection | Attributes work well; cached reflection improves performance |
|
|
| Newtonsoft.Json for criteria serialization | System.Text.Json (recommended) or Newtonsoft.Json | System.Text.Json is built-in with better performance; Newtonsoft.Json works if polymorphism needed |
|
|
|
|
### SqlKata Integration
|
|
|
|
**NuGet Packages:**
|
|
- `SqlKata` - Core query builder
|
|
- `SqlKata.Execution` - Dapper integration for query execution
|
|
|
|
**DI Registration:**
|
|
```csharp
|
|
services.AddScoped<ISearchQueryBuilder, SqlKataSearchQueryBuilder>();
|
|
services.AddScoped<IWorkOrderTraversalService, WorkOrderTraversalService>();
|
|
services.AddSingleton<SqlServerCompiler>();
|
|
```
|
|
|
|
**SqlKata with Dapper Execution:**
|
|
```csharp
|
|
// Build query with SqlKata
|
|
var (sql, parameters) = _queryBuilder.BuildSearchQuery(model);
|
|
|
|
// Execute with Dapper
|
|
var results = await connection.QueryAsync<SearchResult>(sql, parameters);
|
|
```
|
|
|
|
## Design Decisions
|
|
|
|
### Decision: 20-iteration limit for downstream traversal
|
|
|
|
The 20-iteration limit (`@c_MAX_RUNS INT = 20`) for downstream work order traversal is **retained as a fixed value**. Rationale:
|
|
- Manufacturing processes rarely exceed 20 levels of downstream consumption
|
|
- A configurable limit adds complexity without practical benefit
|
|
- The limit protects against runaway queries from circular references
|
|
|
|
### Decision: Circular reference handling
|
|
|
|
Circular references in work order relationships are **handled implicitly** by the MERGE statement's `ON` clause. When a work order is already in `#Temp_WO`, the MERGE updates rather than inserts, preventing infinite loops. No additional detection logic is required.
|
|
|
|
### Decision: MIS non-match results
|
|
|
|
MIS non-match results are **always included** when `ExtractMisData = true`. The non-match data is valuable for quality investigations and should not be optional.
|
|
|
|
### Decision: fn_MatchMIS function availability
|
|
|
|
The `dbo.fn_MatchMIS` table-valued function **must be recreated** in the migrated database. It is a custom SQL Server function that implements MIS record matching logic based on work order routing, master routing, and operation parameters.
|
|
|
|
### Decision: Filter validation location
|
|
|
|
Filter validation (e.g., requiring at least one filter) is enforced **at the UI/API level**, not in the model. This allows the model to represent any valid state while the API layer enforces business rules about minimum filter requirements.
|
|
|
|
### Decision: Large result set handling
|
|
|
|
Large result sets are handled via **streaming to Excel**. The EPPlus library supports streaming writes, and the existing pattern of writing directly to the output stream is preserved. Pagination is not implemented at the search level.
|
|
|
|
---
|
|
|
|
## Codex Review Findings (Spec Accuracy Issues)
|
|
|
|
The following inaccuracies were identified during review and have been addressed in this specification:
|
|
|
|
1. **MIS Extraction Not Filtered**: ~~The spec states MIS extraction joins `#Temp_WO` or `#P_PartOperations` to constrain results.~~ **CORRECTED**: Spec now documents that MIS extraction does NOT join `#Temp_WO`, so work-order/component-lot/item-operation-MIS filters do NOT constrain MIS results. See `QueryTemplate.tt:173,353`.
|
|
|
|
2. **Timespan Filter Requires Both Bounds**: ~~The spec says either min or max boundary enables filtering.~~ **CORRECTED**: Spec now documents that the `LU_WO` timespan filtering requires BOTH min and max values for the combined condition, with separate handling for MIS-only min/max cases. See `QueryTemplate.tt:254,371`, `LotFinderDBExt.cs:157`.
|
|
|
|
3. **WorkOrderComponent Flagged as CARDEX**: ~~The spec implies `PartsList` flag is set for WorkOrderComponent paths.~~ **CORRECTED**: Spec now explicitly states component-lot matches from `WorkOrderComponent` are flagged as CARDEX, not PartsList. See `QueryTemplate.tt:105`.
|
|
|
|
4. **ItemOperationMisFilter Skips WorkOrderTime**: ~~Not documented.~~ **CORRECTED**: Spec now documents that when `ItemOperationMisFilterEnabled` is true, the WorkOrderTime UNION branch is skipped. See `QueryTemplate.tt:258`.
|
|
|
|
5. **InclusionReason Edge Cases**: ~~Spec mentions "Split order" but not "UNKNOWN" case.~~ **CORRECTED**: Spec now documents that `InclusionReason` can return `"UNKNOWN"` when no flags are set. See `SearchResult.cs:152`.
|
|
|
|
6. **Error Handling Not Documented**: Query timeout, debug SQL/Excel writes, and marking search failed on exception are legacy behaviors addressed in the separate error-handling specification. See `LotFinderDBExt.cs:136`, `WorkProcessor.cs:158`.
|