refactor: address code review findings across all projects

Apply comprehensive fixes from code reviews including: - Extract shared utilities (SqlFormatHelper, CellValueConverter, DbDestinationBase) - Add interface abstractions (IAuthenticationService, IDatabaseMigrator, IMisQueryBuilder) - Implement SecureStore for encrypted secrets storage - Fix error handling with proper HTTP status codes and logging - Optimize double enumeration in DevEtlRegistry - Add DataSync.Dev README for developer onboarding - Extract filter panel base classes to reduce duplication - Update code review docs to mark all issues as fixed
2026-01-19 11:05:36 -05:00
parent 08f5aa1447
commit 604bfe919c
148 changed files with 8696 additions and 1538 deletions
@@ -0,0 +1,151 @@
+# Database Status Codes and Enums
+
+This document provides centralized documentation for all status codes and enums used in the JDE Scoping Tool database and application.
+
+## Search Status (SearchStatus Enum)
+
+The `SearchStatus` enum tracks the lifecycle of a search request.
+
+**Source:** `JdeScoping.Core/Models/Enums/SearchStatus.cs`
+
+| Value | Name | Description |
+|-------|------|-------------|
+| 0 | `New` | Search created but not submitted |
+| 1 | `Queued` | Search submitted, waiting for processing |
+| 2 | `Running` | Search actively processing |
+| 3 | `Ended` | Search completed successfully |
+| 4 | `Error` | Search processing failed |
+
+### State Transitions
+
+```
+┌─────────┐    submit    ┌─────────┐   picked up   ┌─────────┐
+│   New   │ ───────────> │ Queued  │ ────────────> │ Running │
+└─────────┘              └─────────┘               └────┬────┘
+                                                        │
+                                             ┌──────────┴──────────┐
+                                             │                     │
+                                        success                  failure
+                                             │                     │
+                                             v                     v
+                                       ┌─────────┐           ┌─────────┐
+                                       │  Ended  │           │  Error  │
+                                       └─────────┘           └─────────┘
+```
+
+### Transitions
+
+| From | To | Trigger |
+|------|----|---------|
+| `New` | `Queued` | User clicks "Run Search" |
+| `Queued` | `Running` | Worker service picks up search |
+| `Running` | `Ended` | Search completes successfully, results stored |
+| `Running` | `Error` | Exception during processing |
+
+### Notes
+
+- Status stored in `Search.Status` column (tinyint)
+- SignalR hub broadcasts status changes in real-time
+- Error details stored in `Search.ErrorMessage` when status = `Error`
+
+---
+
+## Data Update Types (UpdateTypes Enum)
+
+The `UpdateTypes` enum categorizes data synchronization jobs.
+
+**Source:** `JdeScoping.Core/Models/Enums/UpdateTypes.cs`
+
+| Value | Name | Description | Typical Schedule |
+|-------|------|-------------|------------------|
+| 1 | `Hourly` | Incremental delta sync | Every hour |
+| 2 | `Daily` | Daily refresh | Once per day (overnight) |
+| 3 | `Mass` | Full table refresh | Weekly or on-demand |
+
+### Usage
+
+The `DataUpdate` table tracks the last sync time for each entity and update type:
+
+```sql
+SELECT EntityName, UpdateType, LastUpdateDt
+FROM DataUpdate
+WHERE UpdateType = 1  -- Hourly updates
+ORDER BY LastUpdateDt DESC
+```
+
+### Notes
+
+- Value `0` is not defined (avoids default int confusion)
+- Mass updates typically run during maintenance windows
+- Hourly updates capture recent work order completions
+
+---
+
+## JDE Status Codes (StatusCode Lookup Table)
+
+Work order status codes synchronized from JD Edwards (JDE) system.
+
+**Entity:** `JdeScoping.Core/Models/Lookup/StatusCode.cs`
+**Table:** `StatusCode`
+
+### Table Structure
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `Code` | varchar(10) | Status code identifier |
+| `Description` | varchar(100) | Human-readable description |
+| `LastUpdateDate` | int | JDE Julian date of last update |
+| `LastUpdateTime` | int | JDE time of last update |
+
+### Common JDE Status Codes
+
+| Code | Description |
+|------|-------------|
+| 10 | Work Order Created |
+| 20 | Parts Issued |
+| 40 | Work In Progress |
+| 90 | Closed |
+| 95 | Cancelled |
+| 99 | Purged |
+
+> **Note:** Actual codes vary by JDE configuration. The values above are examples; refer to your JDE system administrator for the authoritative list.
+
+### Synchronization
+
+- Source: JDE F0004 / F0005 User Defined Codes tables
+- Sync frequency: Daily
+- Pipeline: `StatusCodePipeline` in DataSync project
+
+---
+
+## Database Column Mappings
+
+### Search Table Status Column
+
+```sql
+CREATE TABLE Search (
+    Id INT PRIMARY KEY,
+    Status TINYINT NOT NULL DEFAULT 0,  -- Maps to SearchStatus enum
+    -- ... other columns
+);
+```
+
+### DataUpdate Table Type Column
+
+```sql
+CREATE TABLE DataUpdate (
+    Id INT PRIMARY KEY,
+    EntityName VARCHAR(100) NOT NULL,
+    UpdateType TINYINT NOT NULL,  -- Maps to UpdateTypes enum
+    LastUpdateDt DATETIME2 NOT NULL,
+    -- ... other columns
+);
+```
+
+---
+
+## See Also
+
+- [SearchStatus.cs](../../NEW/src/JdeScoping.Core/Models/Enums/SearchStatus.cs)
+- [UpdateTypes.cs](../../NEW/src/JdeScoping.Core/Models/Enums/UpdateTypes.cs)
+- [StatusCode.cs](../../NEW/src/JdeScoping.Core/Models/Lookup/StatusCode.cs)
@@ -0,0 +1,117 @@
+# JDE Date Conversion Pattern
+
+## Overview
+
+This document explains the JDE (JD Edwards) date conversion pattern used throughout the data model classes. This pattern appears in 15+ model files and is **intentionally repeated** rather than abstracted—it is required boilerplate for Dapper ORM column mapping.
+
+## JDE Date Format
+
+JD Edwards stores dates in a proprietary **Julian date format** called CYYDDD:
+
+| Component | Description | Example |
+|-----------|-------------|---------|
+| C | Century indicator (0=1900s, 1=2000s) | 1 |
+| YY | Year within century | 25 |
+| DDD | Day of year (001-366) | 019 |
+
+**Example:** January 19, 2025 = `125019`
+- Century: 1 (2000s)
+- Year: 25 (2025)
+- Day: 019 (19th day of year)
+
+JDE also stores time separately as **HHMMSS**:
+- 2:30:45 PM = `143045`
+
+## The Pattern
+
+### Why Private Backing Fields?
+
+Dapper requires a settable property to map database columns. Since we want:
+1. The raw JDE integer values mapped from the database
+2. A clean `DateTime?` exposed to consumers
+3. No public setters on the converted dates
+
+We use this pattern:
+
+```csharp
+using JdeScoping.Core.Helpers;
+
+public class WorkOrder
+{
+    // ... other properties ...
+
+    /// <summary>
+    /// JDE date of last update to record (private backing field for Dapper mapping)
+    /// </summary>
+    private int LastUpdateDate { get; set; }
+
+    /// <summary>
+    /// JDE time of day of last update to record (private backing field for Dapper mapping)
+    /// </summary>
+    private int LastUpdateTime { get; set; }
+
+    /// <summary>
+    /// Timestamp of last update to record (computed from JDE date/time)
+    /// </summary>
+    public DateTime? LastUpdateDt => JdeDateConverter.ToDateTime(LastUpdateDate, LastUpdateTime);
+}
+```
+
+### Why This Cannot Be Abstracted Further
+
+1. **Dapper requires properties on the model class itself** - It cannot map to properties on a composed helper object.
+
+2. **Each model has different date fields** - WorkOrder has `LastUpdateDate/Time`, WorkOrderStep has `EndDate/Time`, etc. The field names vary per entity.
+
+3. **Some models have date-only, some have date+time** - The pattern adapts to each entity's needs.
+
+4. **Private backing fields must match SQL column aliases** - Dapper maps by property name matching column name.
+
+## Centralized Helper
+
+The actual conversion logic **is** centralized in:
+
+```
+NEW/src/JdeScoping.Core/Helpers/JdeDateConverter.cs
+```
+
+This static helper provides:
+- `ToDateTime(int jdeDate)` - Date only conversion
+- `ToDateTime(int jdeDate, int jdeTime)` - Date + time conversion
+- `ToJdeDate(this DateTime)` - Convert DateTime to JDE date
+- `ToJdeTime(this DateTime)` - Convert DateTime to JDE time
+
+## Files Using This Pattern
+
+The pattern appears in these model files:
+- `Models/Inventory/Item.cs`
+- `Models/Inventory/Lot.cs`
+- `Models/Inventory/LotUsage.cs`
+- `Models/Lookup/StatusCode.cs`
+- `Models/Organization/Branch.cs`
+- `Models/Organization/JdeUser.cs`
+- `Models/Organization/OrgHierarchy.cs`
+- `Models/Organization/ProfitCenter.cs`
+- `Models/Organization/RouteMaster.cs`
+- `Models/Organization/WorkCenter.cs`
+- `Models/WorkOrders/WorkOrder.cs`
+- `Models/WorkOrders/WorkOrderComponent.cs`
+- `Models/WorkOrders/WorkOrderRouting.cs`
+- `Models/WorkOrders/WorkOrderStep.cs`
+- `Models/WorkOrders/WorkOrderTime.cs`
+
+## Why Not Records or Init-Only Properties?
+
+Using records with init-only properties would prevent Dapper from setting the values since Dapper uses property setters for mapping. The private setter approach is the cleanest pattern that:
+1. Hides raw JDE integers from consumers
+2. Exposes clean nullable DateTime
+3. Works with Dapper's property-based mapping
+
+## Summary
+
+**This is not code duplication** - it is the minimum required boilerplate for:
+- Database column mapping (private field + Dapper)
+- Type conversion (JdeDateConverter helper)
+- Clean public API (computed DateTime property)
+
+Each repetition is 3 lines of trivial code that cannot be abstracted without breaking Dapper mapping or over-engineering the solution.