Files

T

Joseph Doherty 81b07ce027 feat: extract DevEtl to JdeScoping.DataSync.Dev project

- Create JdeScoping.DataSync.Dev for sandbox testing ETL code
- Create JdeScoping.DataSync.Dev.Tests for associated tests
- Move 22 source files and 8 test files
- Update namespaces from DevEtl to Dev
- Add both projects to solution

2026-01-06 10:18:09 -05:00

4.3 KiB

Raw Blame History

Design: Extract DevEtl to JdeScoping.DataSync.Dev

Date: 2026-01-06 Status: Approved

Purpose

Move dev/testing-specific ETL code from JdeScoping.DataSync into a separate JdeScoping.DataSync.Dev project. This code loads cached JSON/zstd files into SQL Server for local sandbox development and should not be part of the production DataSync assembly.

Scope

Files to Move (Source)

22 files from src/JdeScoping.DataSync/DevEtl/ → src/JdeScoping.DataSync.Dev/

DevEtlRegistry.cs
BranchDevEtl.cs
FunctionCodeDevEtl.cs
ItemDevEtl.cs
JdeUserDevEtl.cs
LotDevEtl.cs
LotUsageCurrDevEtl.cs
LotUsageHistDevEtl.cs
MisDataDevEtl.cs
OrgHierarchyDevEtl.cs
ProfitCenterDevEtl.cs
RouteMasterDevEtl.cs
WorkCenterDevEtl.cs
WorkOrderComponentCurrDevEtl.cs
WorkOrderComponentHistDevEtl.cs
WorkOrderCurrDevEtl.cs
WorkOrderHistDevEtl.cs
WorkOrderRoutingDevEtl.cs
WorkOrderStepCurrDevEtl.cs
WorkOrderStepHistDevEtl.cs
WorkOrderTimeCurrDevEtl.cs
WorkOrderTimeHistDevEtl.cs

Files to Move (Tests)

8 files from tests/JdeScoping.DataSync.Tests/DevEtl/ → tests/JdeScoping.DataSync.Dev.Tests/

BranchDevEtlTests.cs
FunctionCodeDevEtlTests.cs
ItemDevEtlTests.cs
JdeUserDevEtlTests.cs
OrgHierarchyDevEtlTests.cs
ProfitCenterDevEtlTests.cs
RouteMasterDevEtlTests.cs
WorkCenterDevEtlTests.cs

New Project Structure

NEW/
├── src/
│   └── JdeScoping.DataSync.Dev/
│       ├── JdeScoping.DataSync.Dev.csproj
│       ├── DevEtlRegistry.cs
│       ├── BranchDevEtl.cs
│       └── ... (20 more files)
└── tests/
    └── JdeScoping.DataSync.Dev.Tests/
        ├── JdeScoping.DataSync.Dev.Tests.csproj
        ├── BranchDevEtlTests.cs
        └── ... (7 more files)

Dependencies

JdeScoping.DataSync.Dev.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\JdeScoping.DataSync\JdeScoping.DataSync.csproj" />
  </ItemGroup>

</Project>

Reuses ETL infrastructure from DataSync:

EtlPipeline, EtlPipelineBuilder
JsonZstdFileSource
DbBulkImportDestination
JsonColumnSchema

JdeScoping.DataSync.Dev.Tests.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\JdeScoping.DataSync.Dev\JdeScoping.DataSync.Dev.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="..." />
    <PackageReference Include="xunit" Version="..." />
    <PackageReference Include="xunit.runner.visualstudio" Version="..." />
    <PackageReference Include="Shouldly" Version="..." />
    <PackageReference Include="NSubstitute" Version="..." />
  </ItemGroup>

</Project>

Namespace Changes

All source files:

// Before
namespace JdeScoping.DataSync.DevEtl;

// After
namespace JdeScoping.DataSync.Dev;

All test files:

// Before
using JdeScoping.DataSync.DevEtl;

// After
using JdeScoping.DataSync.Dev;

Cleanup

After migration, delete:

src/JdeScoping.DataSync/DevEtl/ (entire folder)
tests/JdeScoping.DataSync.Tests/DevEtl/ (entire folder)

Cache Files

No change to how cache files are located. The DevEtlRegistry continues to accept a cacheDirectory parameter at runtime.

Implementation Steps

Create src/JdeScoping.DataSync.Dev/JdeScoping.DataSync.Dev.csproj
Move 22 source files from DevEtl/ to new project
Update namespace in all source files
Create tests/JdeScoping.DataSync.Dev.Tests/JdeScoping.DataSync.Dev.Tests.csproj
Move 8 test files to new project
Update using statements in test files
Add both projects to solution
Delete old DevEtl/ folders
Build and run tests to verify

4.3 KiB Raw Blame History