feat: extract DevEtl to JdeScoping.DataSync.Dev project

- Create JdeScoping.DataSync.Dev for sandbox testing ETL code - Create JdeScoping.DataSync.Dev.Tests for associated tests - Move 22 source files and 8 test files - Update namespaces from DevEtl to Dev - Add both projects to solution
2026-01-06 10:18:09 -05:00
parent 7ad4e3ec1c
commit 81b07ce027
37 changed files with 2458 additions and 83 deletions
@@ -0,0 +1,162 @@
+# Design: Extract DevEtl to JdeScoping.DataSync.Dev
+
+**Date:** 2026-01-06
+**Status:** Approved
+
+## Purpose
+
+Move dev/testing-specific ETL code from `JdeScoping.DataSync` into a separate `JdeScoping.DataSync.Dev` project. This code loads cached JSON/zstd files into SQL Server for local sandbox development and should not be part of the production DataSync assembly.
+
+## Scope
+
+### Files to Move (Source)
+
+**22 files** from `src/JdeScoping.DataSync/DevEtl/` → `src/JdeScoping.DataSync.Dev/`
+
+- `DevEtlRegistry.cs`
+- `BranchDevEtl.cs`
+- `FunctionCodeDevEtl.cs`
+- `ItemDevEtl.cs`
+- `JdeUserDevEtl.cs`
+- `LotDevEtl.cs`
+- `LotUsageCurrDevEtl.cs`
+- `LotUsageHistDevEtl.cs`
+- `MisDataDevEtl.cs`
+- `OrgHierarchyDevEtl.cs`
+- `ProfitCenterDevEtl.cs`
+- `RouteMasterDevEtl.cs`
+- `WorkCenterDevEtl.cs`
+- `WorkOrderComponentCurrDevEtl.cs`
+- `WorkOrderComponentHistDevEtl.cs`
+- `WorkOrderCurrDevEtl.cs`
+- `WorkOrderHistDevEtl.cs`
+- `WorkOrderRoutingDevEtl.cs`
+- `WorkOrderStepCurrDevEtl.cs`
+- `WorkOrderStepHistDevEtl.cs`
+- `WorkOrderTimeCurrDevEtl.cs`
+- `WorkOrderTimeHistDevEtl.cs`
+
+### Files to Move (Tests)
+
+**8 files** from `tests/JdeScoping.DataSync.Tests/DevEtl/` → `tests/JdeScoping.DataSync.Dev.Tests/`
+
+- `BranchDevEtlTests.cs`
+- `FunctionCodeDevEtlTests.cs`
+- `ItemDevEtlTests.cs`
+- `JdeUserDevEtlTests.cs`
+- `OrgHierarchyDevEtlTests.cs`
+- `ProfitCenterDevEtlTests.cs`
+- `RouteMasterDevEtlTests.cs`
+- `WorkCenterDevEtlTests.cs`
+
+## New Project Structure
+
+```
+NEW/
+├── src/
+│   └── JdeScoping.DataSync.Dev/
+│       ├── JdeScoping.DataSync.Dev.csproj
+│       ├── DevEtlRegistry.cs
+│       ├── BranchDevEtl.cs
+│       └── ... (20 more files)
+└── tests/
+    └── JdeScoping.DataSync.Dev.Tests/
+        ├── JdeScoping.DataSync.Dev.Tests.csproj
+        ├── BranchDevEtlTests.cs
+        └── ... (7 more files)
+```
+
+## Dependencies
+
+### JdeScoping.DataSync.Dev.csproj
+
+```xml
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <TargetFramework>net10.0</TargetFramework>
+    <ImplicitUsings>enable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="..\JdeScoping.DataSync\JdeScoping.DataSync.csproj" />
+  </ItemGroup>
+
+</Project>
+```
+
+Reuses ETL infrastructure from DataSync:
+- `EtlPipeline`, `EtlPipelineBuilder`
+- `JsonZstdFileSource`
+- `DbBulkImportDestination`
+- `JsonColumnSchema`
+
+### JdeScoping.DataSync.Dev.Tests.csproj
+
+```xml
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <TargetFramework>net10.0</TargetFramework>
+    <ImplicitUsings>enable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+    <IsPackable>false</IsPackable>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="..\..\src\JdeScoping.DataSync.Dev\JdeScoping.DataSync.Dev.csproj" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="..." />
+    <PackageReference Include="xunit" Version="..." />
+    <PackageReference Include="xunit.runner.visualstudio" Version="..." />
+    <PackageReference Include="Shouldly" Version="..." />
+    <PackageReference Include="NSubstitute" Version="..." />
+  </ItemGroup>
+
+</Project>
+```
+
+## Namespace Changes
+
+All source files:
+```csharp
+// Before
+namespace JdeScoping.DataSync.DevEtl;
+
+// After
+namespace JdeScoping.DataSync.Dev;
+```
+
+All test files:
+```csharp
+// Before
+using JdeScoping.DataSync.DevEtl;
+
+// After
+using JdeScoping.DataSync.Dev;
+```
+
+## Cleanup
+
+After migration, delete:
+- `src/JdeScoping.DataSync/DevEtl/` (entire folder)
+- `tests/JdeScoping.DataSync.Tests/DevEtl/` (entire folder)
+
+## Cache Files
+
+No change to how cache files are located. The `DevEtlRegistry` continues to accept a `cacheDirectory` parameter at runtime.
+
+## Implementation Steps
+
+1. Create `src/JdeScoping.DataSync.Dev/JdeScoping.DataSync.Dev.csproj`
+2. Move 22 source files from `DevEtl/` to new project
+3. Update namespace in all source files
+4. Create `tests/JdeScoping.DataSync.Dev.Tests/JdeScoping.DataSync.Dev.Tests.csproj`
+5. Move 8 test files to new project
+6. Update using statements in test files
+7. Add both projects to solution
+8. Delete old `DevEtl/` folders
+9. Build and run tests to verify