dohertj2/jdescopingtool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: JDE Scoping Tool migration project

Browse Source 
Set up repository with legacy .NET Framework 4.8 source (OLD/),
new .NET 10 Blazor solution (NEW/), OpenSpec specifications,
documentation, and project configuration.
This commit is contained in:
Joseph Doherty
2026-01-02 07:43:29 -05:00
commit 26ff8d9b4f
1761 changed files with 596509 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/changes/archive/2026-01-01-migrate-database-schema/design.md
+116
View File
@@ -0,0 +1,116 @@
# Database Migration Design
## Overview
This document describes the approach for migrating legacy SQL Server objects to DbUp migration scripts.
## Script Naming Convention
Scripts follow the pattern: `NNN_Description.sql`
```
001_CreateSearchTable.sql (existing)
002_CreateDataUpdateTable.sql (existing)
003_CreateBranchTable.sql
...
025_CreateLotUsageHistTable.sql
026_CreateWorkOrderView.sql
...
032_CreateLastDataUpdatesView.sql
033_CreateWorkOrderFilterParameterType.sql
...
039_CreateItemOperationMisFilterParameterType.sql
040_CreateSubmitSearchProcedure.sql
...
043_CreateResetPartialSearchesProcedure.sql
044_CreateMatchMisFunction.sql
```
### Number Ranges
| Range | Category | Count |
|-------|----------|-------|
| 001-002 | Already done (Search, DataUpdate) | 2 |
| 003-025 | Tables | 23 |
| 026-032 | Views | 7 |
| 033-039 | Table-valued parameter types | 7 |
| 040-043 | Stored procedures | 4 |
| 044 | Functions | 1 |
## Execution Order
Objects must be created in dependency order:
```
1. Reference tables (no FK dependencies)
- Branch, StatusCode, FunctionCode, ProfitCenter, WorkCenter
- Item, JdeUser, OrgHierarchy, RouteMaster, MisData
2. Core tables (depend on reference tables)
- Lot, LotLocation, WorkOrder_Curr, WorkOrder_Hist
- WorkOrderStep_Curr, WorkOrderStep_Hist
- WorkOrderTime_Curr, WorkOrderTime_Hist
- WorkOrderComponent_Curr, WorkOrderComponent_Hist
- LotUsage_Curr, LotUsage_Hist
3. Views (depend on tables)
- Union views: WorkOrder, WorkOrderStep, WorkOrderTime, WorkOrderComponent, LotUsage
- Aggregation views: WorkOrderTotalScrap, LastDataUpdates
4. Types (no dependencies, but used by procedures)
- All 7 TVP types
5. Procedures and Functions (depend on tables, views, types)
- SubmitSearch, StartSearch, CompleteSearch, ResetPartialSearches
- MatchMis function
```
## Schema Mapping
### Data Type Decisions
| Legacy | New | Rationale |
|--------|-----|-----------|
| DATETIME | DATETIME2(7) | Better precision, recommended for new development |
| VARCHAR | VARCHAR | Keep as-is for JDE/CMS compatibility |
| NVARCHAR | NVARCHAR | Keep as-is |
| VARBINARY(MAX) | VARBINARY(MAX) | Keep for Excel storage (per user decision) |
**Note:** The DATETIME → DATETIME2(7) conversion requires updating the database-schema spec to reflect this decision. This is a deliberate modernization choice.
### Index Strategy
- Primary keys: Defined in table creation scripts
- Foreign keys: NOT created (legacy doesn't have them, cache tables)
- Clustered indexes: On primary keys
- Non-clustered indexes: Included in table creation scripts (match legacy exactly)
## DbUp Configuration
The existing `DatabaseMigrator.cs` configuration is appropriate:
- Uses `WithTransaction()` for atomic migrations
- Uses `WithScriptsEmbeddedInAssembly()` for embedded resources
- Uses `EnsureDatabase.For.SqlDatabase()` to create DB if needed
No changes needed to the migrator itself.
## Verification Approach
1. **Script syntax**: Run against local SQL Server container
2. **Object existence**: Query sys.tables, sys.views, sys.procedures
3. **Schema accuracy**: Compare column definitions to legacy
4. **Codex review**: Cross-reference with specs
## File Structure
```
NEW/src/JdeScoping.Database/
├── Scripts/
│ ├── 001_CreateSearchTable.sql (existing)
│ ├── 002_CreateDataUpdateTable.sql (existing)
│ ├── 003_CreateBranchTable.sql (new)
│ ├── ...
│ └── 046_CreateMatchMisFunction.sql (new)
├── DatabaseMigrator.cs (existing, no changes)
└── JdeScoping.Database.csproj (existing, no changes)
```
openspec/changes/archive/2026-01-01-migrate-database-schema/proposal.md
+64
View File
@@ -0,0 +1,64 @@
# Migrate Database Schema
## Summary
Migrate all SQL Server database objects from the legacy .sqlproj to DbUp migration scripts in the new .NET 10 solution. This establishes the data layer foundation for subsequent migration phases.
## Scope
### In Scope
- 25 tables (23 remaining - Search and DataUpdate already migrated)
- 7 views (union views for _Curr/_Hist tables, aggregation views)
- 7 table-valued parameter types (filter parameters for search queries)
- 4 stored procedures (SubmitSearch, StartSearch, CompleteSearch, ResetPartialSearches)
- 1 table-valued function (MatchMis)
### Out of Scope
- Application code changes
- Connection string configuration
- Data migration from existing databases
- Index optimization (will be addressed separately)
## Why
The legacy .sqlproj format is not compatible with .NET 10 and cross-platform development. DbUp provides:
- Version-controlled, sequential migrations
- Idempotent deployments
- Cross-platform compatibility
- Integration with application startup
## What Changes
- **Added**: 44 DbUp migration scripts in `NEW/src/JdeScoping.Database/Scripts/`
- **Modified**: All DATETIME columns converted to DATETIME2(7) for better precision
- **Added**: Spec requirements for DbUp migration patterns and idempotency
## Acceptance Criteria
1. All 44 database objects exist as numbered DbUp migration scripts
2. Scripts run successfully against empty database
3. Scripts are idempotent (can re-run without error)
4. Schema matches legacy database with approved modernizations:
- DATETIME → DATETIME2(7) for better precision
- Non-clustered indexes included in table scripts
5. `openspec validate migrate-database-schema --strict` passes
## Dependencies
- None (this is the foundation phase)
## Risks
| Risk | Mitigation |
|------|------------|
| Schema drift from legacy | Codex MCP review against OLD/Database/ files |
| Missing dependencies between objects | Order scripts by dependency (tables → views → types → procs) |
| Data type mismatches | Use exact types from legacy schema |
## Related Specs
- `database-schema` - Table definitions and relationships
- `sql-views-types` - Views and TVP types
- `sql-business-logic` - Stored procedures and functions
openspec/changes/archive/2026-01-01-migrate-database-schema/specs/database-schema/spec.md
+81
View File
@@ -0,0 +1,81 @@
# Database Schema - Migration Implementation
## ADDED Requirements
### Requirement: Date/time column types
The system SHALL use DATETIME2(7) for all date/time columns instead of legacy DATETIME.
#### Rationale
- DATETIME2(7) provides nanosecond precision vs millisecond for DATETIME
- Larger date range (0001-01-01 to 9999-12-31)
- Recommended for all new SQL Server development
- Compatible with .NET DateTimeOffset
#### Affected Tables
All tables with date/time columns:
- Search (SubmitDT, StartDT, EndDT)
- DataUpdate (UpdateDT)
- WorkOrder_Curr/Hist (various date columns)
- WorkOrderStep_Curr/Hist (LastUpdateDT)
- WorkOrderTime_Curr/Hist (GlDate, LastUpdateDT)
- And all other tables with DATETIME columns
#### Scenario: Date precision preserved
- **WHEN** a datetime value is stored with sub-millisecond precision
- **THEN** the full precision is preserved in DATETIME2(7) columns
## ADDED Requirements
### Requirement: DbUp migration scripts
The system SHALL use DbUp migration scripts to create and maintain the database schema.
#### Migration Script Organization
- Scripts SHALL be numbered sequentially (NNN_Description.sql)
- Scripts SHALL be embedded as resources in JdeScoping.Database assembly
- Scripts SHALL execute in dependency order (tables → views → types → procedures)
#### Script Numbering Ranges
| Range | Category |
|-------|----------|
| 001-025 | Tables |
| 026-032 | Views |
| 033-039 | Table-valued parameter types |
| 040-043 | Stored procedures |
| 044+ | Functions |
#### Scenario: Fresh database deployment
- **WHEN** the application starts against an empty database
- **THEN** DbUp creates all 44 database objects in dependency order
- **AND** the SchemaVersions table records each applied migration
#### Scenario: Incremental migration
- **WHEN** the application starts against a database with some migrations applied
- **THEN** DbUp applies only new migrations not in SchemaVersions
- **AND** existing data is preserved
### Requirement: Migration idempotency
The system SHALL ensure migration scripts are idempotent for safe re-execution.
#### Idempotency Patterns
- Tables: Use `IF NOT EXISTS` checks
- Views: Use `CREATE OR ALTER VIEW`
- Types: Check sys.types before creation
- Procedures: Use `CREATE OR ALTER PROCEDURE`
- Functions: Use `CREATE OR ALTER FUNCTION`
#### Scenario: Re-run migration on existing database
- **WHEN** a migration script runs against a database where the object already exists
- **THEN** the script completes without error
- **AND** the object definition matches the script
openspec/changes/archive/2026-01-01-migrate-database-schema/tasks.md
+210
View File
@@ -0,0 +1,210 @@
# Tasks: Migrate Database Schema
## Phase 0: Foundation Tables (Corrected)
- [x] 001: Search table migration script
- Source: `OLD/Database/Tables/Search.sql`
- Note: Corrected to match legacy schema (ID, UserName, Name, Status, SubmitDT, StartDT, EndDT, Criteria, Results)
- Location: `NEW/src/JdeScoping.Database/Scripts/001_CreateSearchTable.sql`
- [x] 002: DataUpdate table migration script
- Source: `OLD/Database/Tables/DataUpdate.sql`
- Note: Corrected to match legacy schema (SourceSystem, SourceData, TableName, StartDT, EndDT, UpdateType, WasSuccessful, NumberRecords)
- Location: `NEW/src/JdeScoping.Database/Scripts/002_CreateDataUpdateTable.sql`
## Phase 1: Reference Tables (No Dependencies)
- [x] 003: Create Branch table migration script
- Source: `OLD/Database/Tables/Branch.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/003_CreateBranchTable.sql`
- [x] 004: Create StatusCode table migration script
- Source: `OLD/Database/Tables/StatusCode.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/004_CreateStatusCodeTable.sql`
- [x] 005: Create FunctionCode table migration script
- Source: `OLD/Database/Tables/FunctionCode.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/005_CreateFunctionCodeTable.sql`
- [x] 006: Create ProfitCenter table migration script
- Source: `OLD/Database/Tables/ProfitCenter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/006_CreateProfitCenterTable.sql`
- [x] 007: Create WorkCenter table migration script
- Source: `OLD/Database/Tables/WorkCenter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/007_CreateWorkCenterTable.sql`
- [x] 008: Create Item table migration script
- Source: `OLD/Database/Tables/Item.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/008_CreateItemTable.sql`
- [x] 009: Create JdeUser table migration script
- Source: `OLD/Database/Tables/JdeUser.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/009_CreateJdeUserTable.sql`
- [x] 010: Create OrgHierarchy table migration script
- Source: `OLD/Database/Tables/OrgHierarchy.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/010_CreateOrgHierarchyTable.sql`
- [x] 011: Create RouteMaster table migration script
- Source: `OLD/Database/Tables/RouteMaster.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/011_CreateRouteMasterTable.sql`
- [x] 012: Create MisData table migration script
- Source: `OLD/Database/Tables/MisData.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/012_CreateMisDataTable.sql`
## Phase 2: Core Tables (Depend on Reference Tables)
- [x] 013: Create Lot table migration script
- Source: `OLD/Database/Tables/Lot.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/013_CreateLotTable.sql`
- [x] 014: Create LotLocation table migration script
- Source: `OLD/Database/Tables/LotLocation.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/014_CreateLotLocationTable.sql`
- [x] 015: Create WorkOrder_Curr table migration script
- Source: `OLD/Database/Tables/WorkOrder_Curr.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/015_CreateWorkOrderCurrTable.sql`
- [x] 016: Create WorkOrder_Hist table migration script
- Source: `OLD/Database/Tables/WorkOrder_Hist.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/016_CreateWorkOrderHistTable.sql`
- [x] 017: Create WorkOrderStep_Curr table migration script
- Source: `OLD/Database/Tables/WorkOrderStep_Curr.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/017_CreateWorkOrderStepCurrTable.sql`
- [x] 018: Create WorkOrderStep_Hist table migration script
- Source: `OLD/Database/Tables/WorkOrderStep_Hist.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/018_CreateWorkOrderStepHistTable.sql`
- [x] 019: Create WorkOrderTime_Curr table migration script
- Source: `OLD/Database/Tables/WorkOrderTime_Curr.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/019_CreateWorkOrderTimeCurrTable.sql`
- [x] 020: Create WorkOrderTime_Hist table migration script
- Source: `OLD/Database/Tables/WorkOrderTime_Hist.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/020_CreateWorkOrderTimeHistTable.sql`
- [x] 021: Create WorkOrderComponent_Curr table migration script
- Source: `OLD/Database/Tables/WorkOrderComponent_Curr.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/021_CreateWorkOrderComponentCurrTable.sql`
- [x] 022: Create WorkOrderComponent_Hist table migration script
- Source: `OLD/Database/Tables/WorkOrderComponent_Hist.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/022_CreateWorkOrderComponentHistTable.sql`
- [x] 023: Create WorkOrderRouting table migration script
- Source: `OLD/Database/Tables/WorkOrderRouting_Curr.sql`
- Note: File creates `dbo.WorkOrderRouting` (no _Curr suffix, single table)
- Location: `NEW/src/JdeScoping.Database/Scripts/023_CreateWorkOrderRoutingTable.sql`
- [x] 024: Create LotUsage_Curr table migration script
- Source: `OLD/Database/Tables/LotUsage_Curr.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/024_CreateLotUsageCurrTable.sql`
- [x] 025: Create LotUsage_Hist table migration script
- Source: `OLD/Database/Tables/LotUsage_Hist.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/025_CreateLotUsageHistTable.sql`
## Phase 3: Views (Depend on Tables)
- [x] 026: Create WorkOrder view migration script
- Source: `OLD/Database/Views/WorkOrder.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/026_CreateWorkOrderView.sql`
- [x] 027: Create WorkOrderTime view migration script
- Source: `OLD/Database/Views/WorkOrderTime.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/027_CreateWorkOrderTimeView.sql`
- [x] 028: Create WorkOrderStep view migration script
- Source: `OLD/Database/Views/WorkOrderStep.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/028_CreateWorkOrderStepView.sql`
- [x] 029: Create WorkOrderComponent view migration script
- Source: `OLD/Database/Views/WorkOrderComponent.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/029_CreateWorkOrderComponentView.sql`
- [x] 030: Create LotUsage view migration script
- Source: `OLD/Database/Views/LotUsage.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/030_CreateLotUsageView.sql`
- [x] 031: Create WorkOrderTotalScrap view migration script
- Source: `OLD/Database/Views/WorkOrderTotalScrap.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/031_CreateWorkOrderTotalScrapView.sql`
- [x] 032: Create LastDataUpdates view migration script
- Source: `OLD/Database/Views/LastDataUpdates.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/032_CreateLastDataUpdatesView.sql`
## Phase 4: Table-Valued Parameter Types
- [x] 033: Create WorkOrderFilterParameter type migration script
- Source: `OLD/Database/Types/WorkOrderFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/033_CreateWorkOrderFilterParameterType.sql`
- [x] 034: Create ItemNumberFilterParameter type migration script
- Source: `OLD/Database/Types/ItemNumberFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/034_CreateItemNumberFilterParameterType.sql`
- [x] 035: Create ProfitCenterFilterParameter type migration script
- Source: `OLD/Database/Types/ProfitCenterFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/035_CreateProfitCenterFilterParameterType.sql`
- [x] 036: Create WorkCenterFilterParameter type migration script
- Source: `OLD/Database/Types/WorkCenterFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/036_CreateWorkCenterFilterParameterType.sql`
- [x] 037: Create OperatorFilterParameter type migration script
- Source: `OLD/Database/Types/OperatorFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/037_CreateOperatorFilterParameterType.sql`
- [x] 038: Create ComponentLotFilterParameter type migration script
- Source: `OLD/Database/Types/ComponentLotFilterParameter.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/038_CreateComponentLotFilterParameterType.sql`
- [x] 039: Create ItemOperationMisFilterParameter type migration script
- Source: `OLD/Database/Types/ItemOperationMISFilterParameter.sql`
- Note: Using lowercase "Mis" per Codex review naming convention
- Location: `NEW/src/JdeScoping.Database/Scripts/039_CreateItemOperationMisFilterParameterType.sql`
## Phase 5: Stored Procedures
- [x] 040: Create SubmitSearch stored procedure migration script
- Source: `OLD/Database/StoredProcedures/SubmitSearch.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/040_CreateSubmitSearchProcedure.sql`
- [x] 041: Create StartSearch stored procedure migration script
- Source: `OLD/Database/StoredProcedures/StartSearch.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/041_CreateStartSearchProcedure.sql`
- [x] 042: Create CompleteSearch stored procedure migration script
- Source: `OLD/Database/StoredProcedures/CompleteSearch.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/042_CreateCompleteSearchProcedure.sql`
- [x] 043: Create ResetPartialSearches stored procedure migration script
- Source: `OLD/Database/StoredProcedures/ResetPartialSearches.sql`
- Location: `NEW/src/JdeScoping.Database/Scripts/043_CreateResetPartialSearchesProcedure.sql`
## Phase 6: Functions
- [x] 044: Create MatchMis table-valued function migration script
- Source: `OLD/Database/Functions/MatchMis.sql`
- Dependencies: Item, WorkOrder, WorkOrderRouting, RouteMaster, MisData tables
- Location: `NEW/src/JdeScoping.Database/Scripts/044_CreateMatchMisFunction.sql`
## Phase 7: Verification
- [x] 045: Run full migration against clean database
- Validation: All 44 scripts executed in order without error
- Command: `dotnet run --project NEW/src/JdeScoping.Host`
- [x] 046: Verify all objects exist
- Validation: Query sys.tables, sys.views, sys.procedures, sys.types
- Results: 26 tables (25 + SchemaVersions), 7 views, 7 types, 4 procedures, 1 function
- [x] 047: Codex MCP review of migration scripts
- All scripts verified against legacy source during creation
- DATETIME -> DATETIME2(7) conversion applied consistently