- Replace IAuthService with IAuthApiClient for logout functionality
- Add @using JdeScoping.Core.ApiContracts
- LogoutAsync now calls AuthApi.LogoutAsync() and navigates to login
regardless of API result
- Replace IAuthService injection with IAuthApiClient
- Keep ICryptoService for credential encryption
- Add AuthStateProvider injection to notify authentication state
- Use result.Switch() pattern for ApiResult<LoginResultModel> handling
- Properly handle ValidationError with FieldErrors dictionary
Move dev-only JSON reading infrastructure from DataSync to DataSync.Dev:
- JsonColumnSchema (Models/)
- JsonZstdFileSource (Sources/)
- JsonStreamingDataReader (Sources/)
- Utf8JsonStreamingDataReader (Sources/)
Update namespaces and using statements in all DevEtl files.
- Replace ISearchService with ISearchApiClient
- Add @using for JdeScoping.Core.ApiContracts and JdeScoping.Client.Extensions
- Update LoadSearchAsync to use result.Switch() pattern with ApiResult<T>
- Handle CopySearchId, Id, and new search cases with proper error handling
- Use ToClient() extension method to convert Core to Client SearchViewModel
- Add _errorMessage field and display error alert in UI
- Update SubmitSearchInternalAsync and DownloadResultsAsync for consistency
- Add FormatValidationErrors helper for ValidationError.FieldErrors
- Replace ISearchService injection with ISearchApiClient
- Add @using for JdeScoping.Core.ApiContracts and JdeScoping.Client.Extensions
- Update LoadQueueAsync to use result.Switch() pattern with ApiResult<T>
- Add _errorMessage field for error display
- Add RadzenAlert for error message display in UI
- Replace ISearchService injection with ISearchApiClient
- Add @using for JdeScoping.Core.ApiContracts and JdeScoping.Client.Extensions
- Update LoadSearchesAsync to use ApiResult<T>.Switch() pattern
- Add _errorMessage field for error state
- Display RadzenAlert for error conditions
- Use ToClientList() extension method to convert Core->Client view models
- Create JdeScoping.DataSync.Dev for sandbox testing ETL code
- Create JdeScoping.DataSync.Dev.Tests for associated tests
- Move 22 source files and 8 test files
- Update namespaces from DevEtl to Dev
- Add both projects to solution
- Add using statement for JdeScoping.Client.Http namespace
- Register AuthRedirectHandler as a scoped service
- Replace simple HttpClient registration with handler pipeline
- AuthRedirectHandler intercepts 401 responses and redirects to login
- Keep all existing service registrations for backward compatibility
Add integration tests for the Branch development ETL pipeline:
- BranchDevEtlTests with tests for pipeline creation and execution
- Tests verify pipeline creates correctly and loads data from cache files
- Added appsettings.json with local connection string and cache directory
- Added necessary packages (Configuration.Json, SqlClient, Dapper)
Tests require local SQL Server and CACHED_DB_FILES directory with
branch.json.zstd to pass; tests silently skip if resources unavailable.
8-task plan to create comprehensive ETL pipeline documentation:
- Overview, Sources, Transformers, Destinations, Configuration, Troubleshooting
- Update ComponentMap with ETL source paths
- Final verification of links and commits
- Filter MERGE SQL columns to only include columns that exist in destination
(allColumns and updateColumns were using unfiltered source columns)
- Fix schema-qualified table names to use proper [schema].[table] format
instead of wrapping entire name in single brackets
- Add empty column mapping validation to throw early if no columns intersect
- Add JdeDateTransformer output column collision detection in OnInitialize
- Add TODO comment for WithCommandTimeout (stored but not yet passed to
destinations)
- Add tests for FormatQualifiedTableName and output column collision
- Add ParseTableName method to parse table names with optional schema
- Supports: "Table", "dbo.Table", "[dbo].[Table]"
- Returns (schema, table) tuple, defaults to "dbo" schema
- Update DisableIndexes, RebuildIndexes, UpdateStatistics to:
- Use QUOTENAME() for SQL injection protection
- Pass schema and table as parameters via SqlScriptRunner
- Support non-dbo schemas
- Update CustomSql to accept optional parameters and timeout
- Add comprehensive tests for ParseTableName with various formats
- Add DefaultInvalidDateSentinel (1900-01-01) for invalid date handling
- Add optional invalidDateSentinel constructor parameter
- Add MapOrdinal override returning -1 for computed DateTime column
- Add GetDataTypeName override returning "datetime" for computed column
- Update ParseJdeDateTime with comprehensive validation:
- Validate date is positive
- Validate century (0 or 1)
- Validate year (0-99)
- Validate day of year (1-366 and respects leap year)
- Validate time components (hours 0-23, minutes/seconds 0-59)
- Add tests for all new functionality
Add virtual methods to DataTransformerBase for GetBytes, GetChars,
GetData, and GetDataTypeName that properly handle computed columns
by throwing NotSupportedException when MapOrdinal returns -1.
Update TransformingDataReader to delegate these methods to the
transformer instead of directly to the source reader.
Add MapOrdinal method to the IDataTransformer interface and provide
a default implementation in DataTransformerBase. This enables
transformers to report the mapping between transformed ordinals and
source ordinals, supporting use cases like computed columns which
return -1 to indicate no source ordinal.
- Add MapOrdinal(int, IDataReader) to IDataTransformer interface
- Add virtual MapOrdinal implementation in DataTransformerBase
- Add DataTransformerBaseTests with test for default behavior
12-task TDD implementation plan covering:
- MapOrdinal interface and binary method overrides
- ColumnDropTransformer and JdeDateTransformer ordinal mapping
- JDE date validation with 1900-01-01 sentinel
- Column rename collision detection
- SqlScriptRunner parameters support
- CommonScripts with ParseTableName and QUOTENAME
- Destination command timeouts
- Column mapping with destination schema intersection
- EtlPipelineBuilder.WithCommandTimeout validation
- Full test suite verification
- Codex MCP final review
Adds DbQuerySource, an IImportSource implementation that executes SQL
queries against the LotFinder database. Supports parameterized queries
using anonymous objects and configurable command timeouts.
Add transformer that combines JDE Julian date (CYYDDD) and time (HHMMSS)
columns into a single DateTime column. Includes static ParseJdeDateTime
method for direct date conversion.
Joseph Doherty