docs: update documentation for extraction functions migration
- Add ExtractionFunctions.md reference document - Update database-schema spec with 11 extraction functions - Update data-access spec to document extraction function approach - Update search-processing spec with new query builder interface - Add Database.Tests to Testing.md architecture doc - Update DataFlow.md with extraction function flow
This commit is contained in:
@@ -19,6 +19,8 @@ The search flow mirrors the legacy pattern, modernized for ASP.NET Core:
|
||||
|
||||
4. SearchProcessorService (BackgroundService) polls
|
||||
└─> Finds queued searches
|
||||
└─> Passes SearchId to query builder
|
||||
└─> SQL extraction functions read criteria from Search.Criteria JSON
|
||||
└─> Executes query against local cache
|
||||
└─> Generates Excel via ClosedXML
|
||||
└─> Stores result in Search.Results (VARBINARY)
|
||||
|
||||
@@ -121,7 +121,7 @@ The scoping tool cache database includes these primary tables:
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `Search` | User search requests, status, and results (Excel as VARBINARY) |
|
||||
| `Search` | User search requests, status, criteria (JSON), and results (Excel as VARBINARY) |
|
||||
| `DataUpdate` | Tracks last sync timestamp per data type |
|
||||
| `WorkOrder_Curr` | Current work orders from JDE |
|
||||
| `WorkOrder_Hist` | Historical work orders from JDE |
|
||||
@@ -134,6 +134,38 @@ The scoping tool cache database includes these primary tables:
|
||||
| `ProfitCenter` | Profit center reference data |
|
||||
| `SchemaVersions` | DbUp tracking table (auto-created) |
|
||||
|
||||
## Search Criteria Extraction Functions
|
||||
|
||||
The database includes SQL functions that extract filter criteria from the `Search.Criteria` JSON column. These functions enable the query builder to pass only a `SearchId` parameter, with all filter extraction happening in SQL Server.
|
||||
|
||||
### Scalar Functions (3)
|
||||
|
||||
| Function | Returns | Extracts |
|
||||
|----------|---------|----------|
|
||||
| `fn_GetSearchMinimumDt` | DATETIME2 | `$.MinimumDt` |
|
||||
| `fn_GetSearchMaximumDt` | DATETIME2 | `$.MaximumDt` |
|
||||
| `fn_GetSearchExtractMisData` | BIT | `$.ExtractMisData` |
|
||||
|
||||
### Table-Valued Functions (8)
|
||||
|
||||
| Function | Returns | Extracts |
|
||||
|----------|---------|----------|
|
||||
| `fn_GetSearchWorkOrders` | WorkOrderNumber | `$.WorkOrderNumbers` array |
|
||||
| `fn_GetSearchItemNumbers` | ItemNumber | `$.ItemNumbers` array |
|
||||
| `fn_GetSearchProfitCenters` | Code | `$.ProfitCenters` array |
|
||||
| `fn_GetSearchWorkCenters` | Code | `$.WorkCenters` array |
|
||||
| `fn_GetSearchOperatorIDs` | OperatorID | `$.OperatorIDs` array |
|
||||
| `fn_GetSearchComponentLots` | LotNumber, ItemNumber | `$.ComponentLotNumbers` array |
|
||||
| `fn_GetSearchPartOperations` | ItemNumber, OperationNumber, MisNumber, MisRevision | `$.PartOperations` array |
|
||||
|
||||
### Validation Procedure
|
||||
|
||||
| Procedure | Purpose |
|
||||
|-----------|---------|
|
||||
| `usp_ValidateSearchCriteria` | Validates search exists and has valid JSON criteria |
|
||||
|
||||
Error codes: 50001 (not found), 50002 (no criteria), 50003 (invalid JSON)
|
||||
|
||||
## Example Migration Scripts
|
||||
|
||||
### 001_CreateSearchTable.sql
|
||||
|
||||
@@ -5,22 +5,34 @@ The test project uses xUnit for the framework, Shouldly for assertions, and NSub
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
JdeScoping.Tests/
|
||||
├── Unit/
|
||||
│ ├── Services/
|
||||
│ │ ├── SearchServiceTests.cs
|
||||
│ │ ├── ExcelExportServiceTests.cs
|
||||
│ │ └── DataSyncOrchestratorTests.cs
|
||||
│ ├── Repositories/
|
||||
│ │ └── SearchRepositoryTests.cs
|
||||
│ └── Models/
|
||||
│ └── SearchCriteriaTests.cs
|
||||
└── Integration/
|
||||
├── ApiTests/
|
||||
│ ├── SearchControllerTests.cs
|
||||
│ └── LookupControllerTests.cs
|
||||
└── RepositoryTests/
|
||||
└── JdeRepositoryTests.cs
|
||||
tests/
|
||||
├── JdeScoping.Tests/
|
||||
│ ├── Unit/
|
||||
│ │ ├── Services/
|
||||
│ │ │ ├── SearchServiceTests.cs
|
||||
│ │ │ ├── ExcelExportServiceTests.cs
|
||||
│ │ │ └── DataSyncOrchestratorTests.cs
|
||||
│ │ ├── Repositories/
|
||||
│ │ │ └── SearchRepositoryTests.cs
|
||||
│ │ └── Models/
|
||||
│ │ └── SearchCriteriaTests.cs
|
||||
│ └── Integration/
|
||||
│ ├── ApiTests/
|
||||
│ │ ├── SearchControllerTests.cs
|
||||
│ │ └── LookupControllerTests.cs
|
||||
│ └── RepositoryTests/
|
||||
│ └── JdeRepositoryTests.cs
|
||||
├── JdeScoping.Api.IntegrationTests/
|
||||
│ └── ... (API integration tests)
|
||||
└── JdeScoping.Database.Tests/
|
||||
├── Infrastructure/
|
||||
│ └── DatabaseTestBase.cs
|
||||
├── Functions/
|
||||
│ ├── ScalarFunctionTests.cs
|
||||
│ ├── SimpleTableFunctionTests.cs
|
||||
│ └── ComplexTableFunctionTests.cs
|
||||
└── Procedures/
|
||||
└── ValidateSearchCriteriaProcedureTests.cs
|
||||
```
|
||||
|
||||
## Unit Tests
|
||||
@@ -175,7 +187,64 @@ public class SearchRepositoryIntegrationTests : IDisposable
|
||||
}
|
||||
```
|
||||
|
||||
## Database Tests
|
||||
|
||||
The `JdeScoping.Database.Tests` project tests SQL functions and stored procedures against a live SQL Server instance.
|
||||
|
||||
### Test Infrastructure
|
||||
|
||||
Tests inherit from `DatabaseTestBase` which provides:
|
||||
- Connection management via `DatabaseTestFixture`
|
||||
- Automatic test data cleanup
|
||||
- Helper methods for inserting test searches
|
||||
|
||||
```csharp
|
||||
[Collection("DatabaseTests")]
|
||||
public class ScalarFunctionTests : DatabaseTestBase
|
||||
{
|
||||
[Fact]
|
||||
public async Task fn_GetSearchMinimumDt_ValidSearch_ReturnsDateTime()
|
||||
{
|
||||
// Arrange
|
||||
var criteria = new SearchCriteria { MinimumDt = new DateTime(2024, 6, 15) };
|
||||
var searchId = await InsertTestSearchAsync(criteria);
|
||||
|
||||
// Act
|
||||
var result = await Connection.QuerySingleOrDefaultAsync<DateTime?>(
|
||||
"SELECT dbo.fn_GetSearchMinimumDt(@SearchId)",
|
||||
new { SearchId = searchId });
|
||||
|
||||
// Assert
|
||||
result.Should().BeCloseTo(criteria.MinimumDt.Value, TimeSpan.FromSeconds(1));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Test Categories
|
||||
|
||||
| Category | Tests | Description |
|
||||
|----------|-------|-------------|
|
||||
| Scalar Functions | 15 | Test `fn_GetSearchMinimumDt`, `fn_GetSearchMaximumDt`, `fn_GetSearchExtractMisData` |
|
||||
| Simple Table Functions | 38 | Test array extraction for work orders, items, profit centers, work centers, operators |
|
||||
| Complex Table Functions | 23 | Test object extraction for component lots and part operations |
|
||||
| Validation Procedure | 6 | Test `usp_ValidateSearchCriteria` error handling |
|
||||
|
||||
**Total: 82 database tests**
|
||||
|
||||
### Running Database Tests
|
||||
|
||||
```bash
|
||||
# Run all database tests
|
||||
dotnet test tests/JdeScoping.Database.Tests
|
||||
|
||||
# Run specific test class
|
||||
dotnet test tests/JdeScoping.Database.Tests --filter "FullyQualifiedName~ScalarFunctionTests"
|
||||
```
|
||||
|
||||
**Prerequisites**: SQL Server must be running on localhost:1434 with the ScopingTool database.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Core Project](./CoreProject.md)
|
||||
- [Dependencies](./Dependencies.md)
|
||||
- [Database](./Database.md)
|
||||
|
||||
Reference in New Issue
Block a user