docs: update documentation for extraction functions migration

- Add ExtractionFunctions.md reference document
- Update database-schema spec with 11 extraction functions
- Update data-access spec to document extraction function approach
- Update search-processing spec with new query builder interface
- Add Database.Tests to Testing.md architecture doc
- Update DataFlow.md with extraction function flow
This commit is contained in:
Joseph Doherty
2026-01-06 14:54:10 -05:00
parent 35c1e6baf0
commit c6aeb20d9c
8 changed files with 602 additions and 345 deletions
+46 -21
View File
@@ -467,40 +467,65 @@ catch (OracleException ex)
---
### Requirement: Table-valued parameter support
### Requirement: Search criteria extraction via SQL functions
The system SHALL use DataTable for SQL Server table-valued parameters in lookup methods.
The system SHALL use SQL extraction functions to retrieve filter criteria directly from the `Search.Criteria` JSON column.
#### Implementation Pattern
Search query building now uses SearchId to invoke extraction functions rather than passing filter values from C#:
```csharp
public async Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default)
public SearchQueryResult BuildSearchQuery(int searchId)
{
await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
// Query builder generates SQL that calls extraction functions
// Example generated SQL fragment:
// INSERT INTO #P_WorkOrders SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(@SearchId)
// INSERT INTO #P_ItemNumbers SELECT ItemNumber FROM dbo.fn_GetSearchItemNumbers(@SearchId)
var table = new DataTable();
table.Columns.Add("ItemNumber", typeof(string));
foreach (var itemNumber in itemNumbers)
{
table.Rows.Add(itemNumber);
}
var parameters = new DynamicParameters();
parameters.Add("@itemNumbers", table.AsTableValuedParameter("dbo.ItemNumberFilterParameter"));
return (await connection.QueryAsync<Item>(
LotFinderQueries.SQL_LOOKUP_ITEMS,
parameters,
commandTimeout: _options.Value.DefaultTimeoutSeconds))
.AsList();
return new SearchQueryResult(sql, new { SearchId = searchId });
}
```
#### Business Rules
- TVPs SHALL use `AsTableValuedParameter` extension with correct type name
- Query builder SHALL accept only `searchId` parameter (not full criteria object)
- SQL queries SHALL call extraction functions to populate temporary filter tables
- Extraction functions handle JSON parsing and validation in SQL Server
- Invalid JSON or missing criteria results in empty filter sets (no errors thrown)
#### Available Extraction Functions
| Function | Returns |
|----------|---------|
| `fn_GetSearchMinimumDt` | DATETIME2 scalar |
| `fn_GetSearchMaximumDt` | DATETIME2 scalar |
| `fn_GetSearchExtractMisData` | BIT scalar |
| `fn_GetSearchWorkOrders` | WorkOrderNumber table |
| `fn_GetSearchItemNumbers` | ItemNumber table |
| `fn_GetSearchProfitCenters` | Code table |
| `fn_GetSearchWorkCenters` | Code table |
| `fn_GetSearchOperatorIDs` | OperatorID table |
| `fn_GetSearchComponentLots` | LotNumber, ItemNumber table |
| `fn_GetSearchPartOperations` | ItemNumber, OperationNumber, MisNumber, MisRevision table |
#### Scenario: Build search query with extraction functions
- **WHEN** `BuildSearchQuery(123)` is called for a search with work order filter
- **THEN** generated SQL includes `SELECT WorkOrderNumber FROM dbo.fn_GetSearchWorkOrders(123)`
- **AND** only the `@SearchId` parameter is passed to the query
---
### Requirement: Table-valued parameter support for lookups
The system SHALL use DataTable for SQL Server table-valued parameters in reference data lookup methods.
#### Business Rules
- TVPs are used for batch lookups (LookupItemsAsync, LookupWorkordersAsync, etc.)
- TVPs are NOT used for search query execution (replaced by extraction functions)
- DataTable column names SHALL match TVP type column names
- TVPs enable efficient batch lookups with single database round-trip
#### Scenario: Batch lookup with TVP