Commit Graph

3 Commits

Author SHA1 Message Date
Joseph Doherty ead3496cdf refactor: migrate all DevEtl files to protobuf source
Replace JsonZstdFileSource with ProtobufZstdFileSource across all 20
DevEtl pipeline files. This removes the need for manually defined
JsonColumnSchema arrays as protobuf files embed their schema.

Changes per file:
- Remove using JdeScoping.DataSync.Dev.Models
- Remove private JsonColumnSchema[] Schema array
- Change CacheFileName from .json.zstd to .pb.zstd
- Change WithSource from JsonZstdFileSource to ProtobufZstdFileSource
2026-01-06 16:42:56 -05:00
Joseph Doherty 6054412a77 refactor: move JSON ETL classes to DataSync.Dev
Move dev-only JSON reading infrastructure from DataSync to DataSync.Dev:
- JsonColumnSchema (Models/)
- JsonZstdFileSource (Sources/)
- JsonStreamingDataReader (Sources/)
- Utf8JsonStreamingDataReader (Sources/)

Update namespaces and using statements in all DevEtl files.
2026-01-06 10:25:45 -05:00
Joseph Doherty 81b07ce027 feat: extract DevEtl to JdeScoping.DataSync.Dev project
- Create JdeScoping.DataSync.Dev for sandbox testing ETL code
- Create JdeScoping.DataSync.Dev.Tests for associated tests
- Move 22 source files and 8 test files
- Update namespaces from DevEtl to Dev
- Add both projects to solution
2026-01-06 10:18:09 -05:00