Commit Graph

258 Commits

Author SHA1 Message Date
Joseph Doherty 9bf0c29add refactor(configmanager): simplify SecureStore UI with unified info view
Consolidate SecureStoreLockedFormView and SecureStoreUnlockedFormView into
a single SecureStoreInfoFormView that displays store status and metadata.
Simplifies MainWindowViewModel by removing redundant state management.
Also adds design docs for RegexTransformer feature.
2026-01-22 09:40:38 -05:00
Joseph Doherty 5669bac221 test(configmanager): add RegexTransformerViewModel unit tests
Tests cover:
- Loading from model
- Serializing to model
- Test pattern command (success and error cases)
- Mode switching and help text
- Summary display
- Change notification
2026-01-22 07:23:19 -05:00
Joseph Doherty b9c977dea1 feat(configmanager): register RegexEditorView DataTemplate in MainWindow 2026-01-22 07:22:08 -05:00
Joseph Doherty 4954368284 feat(configmanager): add RegexEditorView Avalonia UI
Implements editor with:
- Column name input
- Mode toggle (Find & Replace / Match & Extract)
- Pattern and Replacement inputs
- Case insensitive checkbox
- NonMatchBehavior dropdown
- Live test/preview section with result display
- Pattern examples help box
2026-01-22 07:21:11 -05:00
Joseph Doherty 6095f3dca0 feat(configmanager): register Regex transformer in TransformerFactory
Add Regex to:
- Create() factory method
- CreateNew() factory method
- AvailableTypes list
2026-01-22 07:19:01 -05:00
Joseph Doherty 5058cd6802 feat(configmanager): add RegexTransformerViewModel
Implements ViewModel for Regex transformer editor with:
- Column, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
- Mode toggle between Find & Replace and Match & Extract
- Live test/preview functionality with error handling
2026-01-22 07:17:53 -05:00
Joseph Doherty 8734c57c9b test(datasync): add edge case tests for RegexTransformer
Tests cover:
- Capture group substitution in replacement
- Case-insensitive matching
- Null/DBNull passthrough
- Non-target columns unchanged
- Invalid regex pattern handling
- Column not found handling
2026-01-22 07:16:25 -05:00
Joseph Doherty af73aeba77 test(datasync): add Match & Extract mode tests for RegexTransformer
Tests cover:
- Extracting first capture group
- NonMatchBehavior: KeepOriginal, ReturnNull, ReturnEmpty
2026-01-22 07:15:06 -05:00
Joseph Doherty cb41e42bb7 feat(datasync): add RegexTransformer with Find & Replace mode
Initial implementation supporting:
- Find & Replace mode with regex pattern and replacement string
- Case-insensitive option
- NonMatchBehavior enum for handling non-matches
2026-01-22 07:14:08 -05:00
Joseph Doherty f87805b931 feat(configmanager): add NonMatchBehavior enum and regex properties to TransformerModel
Add configuration model support for the new Regex transformer including:
- NonMatchBehavior enum with JSON string serialization
- ColumnName, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
2026-01-22 07:12:22 -05:00
Joseph Doherty 6642c83cdb feat(configmanager): add runtime config validation using Infrastructure validators
Enable ConfigManager to validate runtime configuration (SecureStore secrets,
connection strings, LDAP) using the same validators as the Host application.
Adds AddInfrastructureValidators() extension for shared validator registration.
2026-01-21 18:31:42 -05:00
Joseph Doherty e5fe2f06e9 feat: add startup config validation and document ConfigManager pipeline editor
Add ConfigurationValidationRunner with IConfigurationValidator interface for
validating required settings at startup. Includes SecureStore and LDAP validators.
Expand ConfigManager with pipeline editing UI, dialogs, and step editors.
Update documentation with config validation guidance.
2026-01-21 17:47:15 -05:00
Joseph Doherty ceb63bfefb refactor(webui): remove pipeline viewer feature
Remove the read-only pipeline viewer from the web UI:
- Delete PipelineViewer.razor page and supporting components
- Delete PipelineController and PipelineMapper from API
- Delete Pipeline DTOs from Core
- Delete PipelineApiClient from Client
- Remove navigation link and DI registrations
- Delete obsolete plan documents

The ConfigManager utility retains pipeline editing capabilities.
2026-01-21 10:14:43 -05:00
Joseph Doherty 94d5a864e0 feat(configmanager): integrate SecureStore for credential management
Add SecureStore integration to ConfigManager for secure handling of connection
strings and sensitive configuration values. Includes store/secret management
UI, encrypted .store file support, and comprehensive test coverage.
2026-01-20 02:51:16 -05:00
Joseph Doherty d49330e697 docs: add XML documentation and ConfigManager implementation plans
Add comprehensive XML documentation (param/returns tags) across 132 source
files to improve IntelliSense and API discoverability. Include ConfigManager
design documents and implementation plans for phases 1-9.
2026-01-20 02:26:26 -05:00
Joseph Doherty c044337539 feat(configmanager): complete service registrations in App.axaml.cs
Add IDialogService registration using factory pattern for window access.
Add GetMainWindow helper method to provide window reference for dialogs.
2026-01-19 19:59:34 -05:00
Joseph Doherty 2d33e44067 feat(configmanager): add DataTemplates for form view automatic selection
Add Window.DataTemplates section to MainWindow.axaml to enable automatic
view selection when SelectedFormViewModel is set. Each form ViewModel
type is now mapped to its corresponding View.
2026-01-19 19:59:19 -05:00
Joseph Doherty 700d62ee74 feat(configmanager): add XAML form views for all configuration sections
Add 7 form view pairs implementing the dark theme UI for ConfigManager:
- DataSyncFormView: Sync intervals, performance, and retention settings
- DataAccessFormView: Query timeouts and schema name configuration
- AuthFormView: Cookie name and expiration settings
- LdapFormView: Server URLs, directory structure, and dev options
- SearchFormView: Result limits and timeout configuration
- ExcelExportFormView: Sheet protection, format, timezone, and debug options
- PipelineFormView: Source/Destination/Schedules with collapsible Expanders

All views follow the dark theme colors from configmanager-ui-design.md:
- Background layers: #0D0F12, #151920, #232A35
- Text hierarchy: #E6EDF5, #9BA8B8, #5C6A7A
- Borders: #2D3540, #3D4550
- Accent: #5C9AFF

Task 25 of docs/plans/2026-01-19-configmanager-phases7-9.md
2026-01-19 19:57:51 -05:00
Joseph Doherty 042e036c35 feat(configmanager): wire up form selection in MainWindowViewModel
- Add IFileSystem and IDialogService dependencies to constructor
- Implement OnSelectedNodeChanged to create appropriate form ViewModels
- Add LoadConfigForTesting helper for unit testing
- Add MarkAsChanged helper to track unsaved changes
- Update OpenFolderAsync to use IDialogService
- Add comprehensive unit tests for form selection
2026-01-19 19:57:27 -05:00
Joseph Doherty 46e94539cd feat(configmanager): add DiffPreviewDialog
Add DiffPreviewDialog and DiffPreviewDialogViewModel to display a diff
preview of configuration changes before saving. The dialog shows line
numbers, insertions (green), deletions (red), and unchanged lines with
a dark theme matching the ConfigManager design spec.
2026-01-19 19:52:38 -05:00
Joseph Doherty 7dd4c46cb7 feat(configmanager): add IDialogService interface and AvaloniaDialogService
Add dialog service abstraction for platform-specific dialogs:
- IDialogService interface with folder picker, message, confirmation,
  diff preview, and validation results methods
- AvaloniaDialogService implementation using MsBox.Avalonia
- Basic implementations for ShowDiffPreviewAsync and ShowValidationResultsAsync
  (full dialogs to be implemented in Tasks 22-23)
- Add MessageBox.Avalonia package reference
2026-01-19 19:52:30 -05:00
Joseph Doherty c3684f5150 feat(configmanager): add PipelineFormViewModel and ScheduleFormViewModel
Add form ViewModels for editing pipeline configurations in the ConfigManager.
ScheduleFormViewModel wraps ScheduleModel for schedule editing.
PipelineFormViewModel wraps PipelineModel with schedule sub-ViewModels.
2026-01-19 19:50:20 -05:00
Joseph Doherty 6e2decd21f feat(configmanager): add LdapFormViewModel
Implement LdapFormViewModel for editing LDAP configuration section with
properties for ServerUrlsText, GroupDn, SearchBase, ConnectionTimeoutSeconds,
UseFakeAuth, and AdminBypassUsersText. Array properties use newline-separated
text with StringSplitOptions.RemoveEmptyEntries | TrimEntries for splitting.
2026-01-19 19:45:43 -05:00
Joseph Doherty cc555e4e34 feat(configmanager): add SearchFormViewModel
Implements Task 18 from phases 7-9 plan. SearchFormViewModel wraps
SearchSection model with properties for MaxResultRows, TimeoutSeconds,
and MaxConcurrentSearches. Includes full test coverage with 7 tests
verifying initialization, two-way binding, change notification, and
null argument handling.
2026-01-19 19:45:07 -05:00
Joseph Doherty 64518c7000 feat(configmanager): add DataAccessFormViewModel 2026-01-19 19:42:10 -05:00
Joseph Doherty c416544efe feat(configmanager): add DataSyncFormViewModel
Add form ViewModel that wraps DataSyncSection model for two-way binding
with change tracking in the ConfigManager UI.
2026-01-19 19:40:31 -05:00
Joseph Doherty c22e2ed877 feat(configmanager): add MainWindowViewModel with service wiring
Complete MainWindowViewModel implementation with:
- Constructor injection for IConfigFileService, IValidationService,
  IBackupService, IAutoDiscoveryService, and ILogger
- Properties: ConfigFolderPath, HasUnsavedChanges, ValidationStatus,
  ValidationStatusColor, SelectedNode, SelectedFormViewModel, TreeNodes
- Commands: OpenFolderCommand, SaveCommand, ExitCommand, UndoCommand,
  RedoCommand, ValidateCommand, TestConnectionCommand
- Methods: InitializeAsync, OpenFolderAsync, LoadConfigAsync,
  BuildTreeNodes, OnSelectedNodeChanged, SaveAsync, Validate,
  TestConnectionAsync

Update App.axaml.cs to register all services in DI container:
- IFileSystem -> FileSystem (singleton)
- IAutoDiscoveryService -> AutoDiscoveryService (singleton)
- IBackupService -> BackupService (singleton)
- IDiffService -> DiffService (singleton)
- IValidationService -> ValidationService (singleton)
- IConfigFileService -> ConfigFileService (scoped)
- MainWindowViewModel (transient)

Wire MainWindow.DataContext to resolved MainWindowViewModel.
2026-01-19 17:53:25 -05:00
Joseph Doherty 8caaa8ee54 feat(configmanager): add TreeNodeViewModel 2026-01-19 17:51:15 -05:00
Joseph Doherty e9fc764650 feat(configmanager): add MainWindow view 2026-01-19 17:49:04 -05:00
Joseph Doherty 9677b751e6 feat(configmanager): add AutoDiscoveryService with tests
Add service for auto-discovering configuration file locations. The
service searches in prioritized order:
1. JDESCOPING_CONFIG_PATH environment variable
2. Same directory as executable
3. ../JdeScoping.Host/ relative to executable
4. User config directory (~/.jdescoping on Unix, %LOCALAPPDATA%\JdeScoping on Windows)

Includes 9 unit tests covering all search locations, priority order,
edge cases (missing directory, missing appsettings.json), and
cancellation token support.
2026-01-19 17:47:03 -05:00
Joseph Doherty 68da728cdf feat(configmanager): add DiffService with tests 2026-01-19 17:44:28 -05:00
Joseph Doherty c8f3c0060d feat(configmanager): add ValidationService with tests 2026-01-19 17:42:14 -05:00
Joseph Doherty 0e441898a6 feat(configmanager): add BackupService with tests 2026-01-19 17:40:22 -05:00
Joseph Doherty 54620ccb2e feat(configmanager): add ConfigFileService with tests
Add config file loading and saving service following TDD approach:
- IConfigFileService interface for loading/saving config files
- ConfigLoadException for descriptive error handling
- ConfigFileService implementation with JSON serialization
- Unit tests with mocked IFileSystem dependency
2026-01-19 17:38:20 -05:00
Joseph Doherty 4335286560 feat(configmanager): add configuration models 2026-01-19 17:36:10 -05:00
Joseph Doherty 0e1bb6aa35 feat(configmanager): add MVVM base classes 2026-01-19 17:34:53 -05:00
Joseph Doherty c055bc6c78 feat(configmanager): add IFileSystem abstraction
Add file system abstraction to enable testability for file operations.
- IFileSystem interface with common file operations
- FileSystem implementation wrapping System.IO
- Unit tests for FileExists functionality
2026-01-19 17:33:39 -05:00
Joseph Doherty 54d4fd0eb6 feat(configmanager): add test project 2026-01-19 17:31:23 -05:00
Joseph Doherty 4ab17259d6 feat(configmanager): create initial project structure 2026-01-19 17:29:08 -05:00
Joseph Doherty fbe58a81e4 refactor(securestoremanager): add platform service abstractions and constants
Implement deferred code review findings:
- Add IDialogService/IClipboardService interfaces for testable platform operations
- Create AvaloniaDialogService and AvaloniaClipboardService implementations
- Extract dialog strings and file extensions to centralized Constants classes
- Refactor ViewModels to use DI instead of event delegates
- Update tests to use mock services
2026-01-19 16:54:35 -05:00
Joseph Doherty 1c546c111a fix: resolve test failures from timezone conversion and interface rename
- Fix CriteriaSheetGenerator.FormatTimestamp to handle all DateTimeKind values
- Update TestWebApplicationFactory to use IAuthenticationService
- Add logger parameter to ExcelParserServiceTests
- Add SecureStoreManager to solution under /utils/ folder
2026-01-19 14:55:22 -05:00
Joseph Doherty 604bfe919c refactor: address code review findings across all projects
Apply comprehensive fixes from code reviews including:
- Extract shared utilities (SqlFormatHelper, CellValueConverter, DbDestinationBase)
- Add interface abstractions (IAuthenticationService, IDatabaseMigrator, IMisQueryBuilder)
- Implement SecureStore for encrypted secrets storage
- Fix error handling with proper HTTP status codes and logging
- Optimize double enumeration in DevEtlRegistry
- Add DataSync.Dev README for developer onboarding
- Extract filter panel base classes to reduce duplication
- Update code review docs to mark all issues as fixed
2026-01-19 11:05:36 -05:00
Joseph Doherty 08f5aa1447 docs: add MisData archival implementation and unit test report
Document the MisData_Curr/Hist table split in architecture docs, add
post-sync archival scripts to move BackLevel records to history table,
and generate comprehensive unit test report (856 tests, 100% pass rate).
2026-01-19 04:03:25 -05:00
Joseph Doherty 1e5b1bb3da feat(database): split MisData into archival tables
Split MisData table into MisData_Curr (Status='Current') and
MisData_Hist (Status='BackLevel') following existing archival pattern.

- Add MisData_Curr and MisData_Hist table scripts
- Create MisData view (UNION ALL) for backward compatibility
- Update production and DEV pipeline configurations
- Update unit tests for new table count (22 tables)
- Update database documentation
2026-01-19 01:08:37 -05:00
Joseph Doherty 5a798b089b feat(excelio): allow DeleteRows on protected worksheets
Enable users to delete rows in protected Excel worksheets for better
data management flexibility.
2026-01-19 00:44:13 -05:00
Joseph Doherty 0c8657713b refactor(core): reorganize DTOs into Models and ViewModels folders
Move DTOs from ApiContracts to appropriate locations:
- SignalR DTOs → ViewModels (renamed Dto→ViewModel suffix)
- Pipeline DTOs → Models/Pipelines
- UserInfoDto → Models/Auth
- DataUpdateDto → Models/Infrastructure
2026-01-19 00:34:57 -05:00
Joseph Doherty 7e36bb4225 refactor: remove unused classes and consolidate ViewModels in Core
Remove 9 unused types from Core (duplicate extension classes, TableSpec, ColumnSpec, LotLocation), move ComponentLotViewModel and OperatorViewModel from Client to Core, and refactor DataSync.Dev to use pipeline-based configuration. Fix Login.razor to use UserInfoDto directly.
2026-01-19 00:13:12 -05:00
Joseph Doherty 80057590f4 feat(client): add pipeline viewer to navigation
Add navigation link to the ETL Pipeline Viewer page in the main navigation
bar, providing easy access for users to monitor and inspect pipeline
configurations and execution status.
2026-01-07 08:25:25 -05:00
Joseph Doherty dc927e2f71 feat(client): add PipelineViewer page
Add admin page for viewing ETL pipeline configurations with:
- Pipeline selector dropdown (alphabetical list)
- Status summary table (type, last run, success, next required, status)
- Execution history table with paging (10 per page)
- Source, destination, and scripts info cards
- Three PipelineScheduleSection components for Mass, Daily, Hourly schedules
- SQL modal integration for viewing queries and scripts
2026-01-07 08:18:57 -05:00
Joseph Doherty 2a15028e00 feat(client): add PipelineApiClient and admin components
- Add IPipelineApiClient interface in Core ApiContracts
- Add PipelineApiClient implementation extending ApiClientBase
- Create Components/Admin directory for admin UI components
- Add SqlQueryModal component for displaying SQL queries with copy-to-clipboard
- Add PipelineScheduleSection component for pipeline schedule display
- Register IPipelineApiClient in Program.cs DI container
- Add Admin components namespace to _Imports.razor
2026-01-07 08:14:37 -05:00