Initial commit: JDE Scoping Tool migration project
Set up repository with legacy .NET Framework 4.8 source (OLD/), new .NET 10 Blazor solution (NEW/), OpenSpec specifications, documentation, and project configuration.
This commit is contained in:
@@ -0,0 +1,84 @@
|
||||
# Data Flow
|
||||
|
||||
The system has two primary data flows: search execution and data synchronization.
|
||||
|
||||
## Search Flow
|
||||
|
||||
The search flow mirrors the legacy pattern, modernized for ASP.NET Core:
|
||||
|
||||
```
|
||||
1. User submits search via Blazor UI
|
||||
└─> POST /api/search (SearchCriteria JSON)
|
||||
|
||||
2. SearchController validates and stores in SQL Server
|
||||
└─> Search record created with Status = "Queued"
|
||||
└─> Returns SearchId to client
|
||||
|
||||
3. Client connects to SignalR StatusHub
|
||||
└─> Subscribes to updates for their SearchId
|
||||
|
||||
4. SearchProcessorService (BackgroundService) polls
|
||||
└─> Finds queued searches
|
||||
└─> Executes query against local cache
|
||||
└─> Generates Excel via ClosedXML
|
||||
└─> Stores result in Search.Results (VARBINARY)
|
||||
└─> Updates Status = "Complete"
|
||||
|
||||
5. StatusHub pushes update to client
|
||||
└─> Client shows "Complete" status
|
||||
|
||||
6. User clicks download
|
||||
└─> GET /api/search/{id}/download
|
||||
└─> Returns Excel file stream
|
||||
```
|
||||
|
||||
## Search Status States
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| `Queued` | Search submitted, waiting for processing |
|
||||
| `Processing` | Background service is executing the search |
|
||||
| `Generating` | Query complete, generating Excel file |
|
||||
| `Complete` | Excel ready for download |
|
||||
| `Failed` | Error occurred during processing |
|
||||
|
||||
## Data Sync Flow
|
||||
|
||||
The `DataSyncService` runs on a schedule to keep the local SQL Server cache current:
|
||||
|
||||
```
|
||||
DataSyncService runs on schedule:
|
||||
├── Mass refresh: Full reload (weekly or manual trigger)
|
||||
├── Daily refresh: Last 24-48 hours of changes
|
||||
└── Hourly refresh: Incremental updates
|
||||
|
||||
Each sync:
|
||||
1. Determine tables to sync based on schedule
|
||||
2. Query JDE/CMS Oracle for changes since last sync
|
||||
3. Bulk insert/update to SQL Server cache
|
||||
4. Update DataUpdate table with timestamp
|
||||
```
|
||||
|
||||
## Sync Schedules
|
||||
|
||||
| Schedule | Frequency | Scope |
|
||||
|----------|-----------|-------|
|
||||
| Mass | Weekly (Sunday 2 AM) or manual | Full reload of all cached tables |
|
||||
| Daily | Daily (3 AM) | Changes from last 48 hours |
|
||||
| Hourly | Every hour | Incremental changes since last sync |
|
||||
|
||||
The schedules are configured via cron expressions in `appsettings.json` and parsed using the Cronos library.
|
||||
|
||||
## Database Connections
|
||||
|
||||
| Database | Purpose | Driver |
|
||||
|----------|---------|--------|
|
||||
| SQL Server | Local cache, search storage | Microsoft.Data.SqlClient |
|
||||
| JDE Oracle | Enterprise work order data | Oracle.ManagedDataAccess.Core |
|
||||
| CMS Oracle | Enterprise CMS data (migrated from Sybase) | Oracle.ManagedDataAccess.Core |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Host Project](./HostProject.md)
|
||||
- [Core Project](./CoreProject.md)
|
||||
- [Configuration](./Configuration.md)
|
||||
Reference in New Issue
Block a user