# Design: Extract DevEtl to JdeScoping.DataSync.Dev

**Date:** 2026-01-06
**Status:** Approved

## Purpose

Move dev/testing-specific ETL code from `JdeScoping.DataSync` into a separate `JdeScoping.DataSync.Dev` project. This code loads cached JSON/zstd files into SQL Server for local sandbox development and should not be part of the production DataSync assembly.

## Scope

### Files to Move (Source)

**22 files** from `src/JdeScoping.DataSync/DevEtl/` → `src/JdeScoping.DataSync.Dev/`

- `DevEtlRegistry.cs`
- `BranchDevEtl.cs`
- `FunctionCodeDevEtl.cs`
- `ItemDevEtl.cs`
- `JdeUserDevEtl.cs`
- `LotDevEtl.cs`
- `LotUsageCurrDevEtl.cs`
- `LotUsageHistDevEtl.cs`
- `MisDataDevEtl.cs`
- `OrgHierarchyDevEtl.cs`
- `ProfitCenterDevEtl.cs`
- `RouteMasterDevEtl.cs`
- `WorkCenterDevEtl.cs`
- `WorkOrderComponentCurrDevEtl.cs`
- `WorkOrderComponentHistDevEtl.cs`
- `WorkOrderCurrDevEtl.cs`
- `WorkOrderHistDevEtl.cs`
- `WorkOrderRoutingDevEtl.cs`
- `WorkOrderStepCurrDevEtl.cs`
- `WorkOrderStepHistDevEtl.cs`
- `WorkOrderTimeCurrDevEtl.cs`
- `WorkOrderTimeHistDevEtl.cs`

### Files to Move (Tests)

**8 files** from `tests/JdeScoping.DataSync.Tests/DevEtl/` → `tests/JdeScoping.DataSync.Dev.Tests/`

- `BranchDevEtlTests.cs`
- `FunctionCodeDevEtlTests.cs`
- `ItemDevEtlTests.cs`
- `JdeUserDevEtlTests.cs`
- `OrgHierarchyDevEtlTests.cs`
- `ProfitCenterDevEtlTests.cs`
- `RouteMasterDevEtlTests.cs`
- `WorkCenterDevEtlTests.cs`

## New Project Structure

```
NEW/
├── src/
│   └── JdeScoping.DataSync.Dev/
│       ├── JdeScoping.DataSync.Dev.csproj
│       ├── DevEtlRegistry.cs
│       ├── BranchDevEtl.cs
│       └── ... (20 more files)
└── tests/
    └── JdeScoping.DataSync.Dev.Tests/
        ├── JdeScoping.DataSync.Dev.Tests.csproj
        ├── BranchDevEtlTests.cs
        └── ... (7 more files)
```

## Dependencies

### JdeScoping.DataSync.Dev.csproj

```xml
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\JdeScoping.DataSync\JdeScoping.DataSync.csproj" />
  </ItemGroup>

</Project>
```

Reuses ETL infrastructure from DataSync:
- `EtlPipeline`, `EtlPipelineBuilder`
- `JsonZstdFileSource`
- `DbBulkImportDestination`
- `JsonColumnSchema`

### JdeScoping.DataSync.Dev.Tests.csproj

```xml
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\JdeScoping.DataSync.Dev\JdeScoping.DataSync.Dev.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="..." />
    <PackageReference Include="xunit" Version="..." />
    <PackageReference Include="xunit.runner.visualstudio" Version="..." />
    <PackageReference Include="Shouldly" Version="..." />
    <PackageReference Include="NSubstitute" Version="..." />
  </ItemGroup>

</Project>
```

## Namespace Changes

All source files:
```csharp
// Before
namespace JdeScoping.DataSync.DevEtl;

// After
namespace JdeScoping.DataSync.Dev;
```

All test files:
```csharp
// Before
using JdeScoping.DataSync.DevEtl;

// After
using JdeScoping.DataSync.Dev;
```

## Cleanup

After migration, delete:
- `src/JdeScoping.DataSync/DevEtl/` (entire folder)
- `tests/JdeScoping.DataSync.Tests/DevEtl/` (entire folder)

## Cache Files

No change to how cache files are located. The `DevEtlRegistry` continues to accept a `cacheDirectory` parameter at runtime.

## Implementation Steps

1. Create `src/JdeScoping.DataSync.Dev/JdeScoping.DataSync.Dev.csproj`
2. Move 22 source files from `DevEtl/` to new project
3. Update namespace in all source files
4. Create `tests/JdeScoping.DataSync.Dev.Tests/JdeScoping.DataSync.Dev.Tests.csproj`
5. Move 8 test files to new project
6. Update using statements in test files
7. Add both projects to solution
8. Delete old `DevEtl/` folders
9. Build and run tests to verify