Joseph Doherty
26ff8d9b4f
Set up repository with legacy .NET Framework 4.8 source (OLD/), new .NET 10 Blazor solution (NEW/), OpenSpec specifications, documentation, and project configuration.
157 lines
7.6 KiB
Markdown
157 lines
7.6 KiB
Markdown
# Data Sync Specification - Implementation Additions
|
|
|
|
## Purpose
|
|
|
|
This specification extends the base data-sync spec with additional implementation-focused requirements for the BackgroundService pattern and parallel fetch isolation.
|
|
|
|
## ADDED Requirements
|
|
|
|
### Requirement: Background service implementation pattern
|
|
|
|
The system SHALL implement the data synchronization service following .NET BackgroundService best practices for hosted service lifecycle management.
|
|
|
|
#### Inputs
|
|
|
|
- `IServiceScopeFactory` for creating scoped service instances
|
|
- `IOptions<DataSyncOptions>` for configuration access
|
|
- `ILogger<DataSyncService>` for structured logging
|
|
- `CancellationToken` from `ExecuteAsync` stoppingToken parameter
|
|
|
|
#### Outputs
|
|
|
|
- Continuously running background task that checks schedules and executes syncs
|
|
- Proper cleanup on shutdown with all resources disposed
|
|
- Logging scope context for all operations
|
|
|
|
#### Business Rules
|
|
|
|
- The service MUST implement `BackgroundService.ExecuteAsync(CancellationToken)`
|
|
- The main loop MUST use `Task.Delay(checkInterval, stoppingToken)` between cycles
|
|
- Each sync cycle MUST create a new `IServiceScope` via `IServiceScopeFactory.CreateAsyncScope()`
|
|
- All scoped services MUST be resolved from the current scope, not from root provider
|
|
- The scope MUST be disposed using `await using` pattern after each cycle
|
|
- Exception handling MUST catch and log errors without crashing the service
|
|
- `OperationCanceledException` MUST be caught and result in graceful loop exit when `stoppingToken.IsCancellationRequested`
|
|
- The service MUST NOT use static state or shared mutable collections
|
|
|
|
#### Scenario: Normal sync cycle execution
|
|
|
|
- **WHEN** the BackgroundService enters ExecuteAsync
|
|
- **THEN** the service SHALL call CloseOpenUpdateEntriesAsync to recover from prior crashes
|
|
- **THEN** the service SHALL enter a while loop checking `!stoppingToken.IsCancellationRequested`
|
|
- **THEN** each iteration SHALL create a new IServiceScope
|
|
- **THEN** the ISyncOrchestrator SHALL be resolved from the scope
|
|
- **THEN** ExecutePendingSyncsAsync SHALL be called with the stoppingToken
|
|
- **THEN** the scope SHALL be disposed after the call completes
|
|
- **THEN** Task.Delay SHALL pause before the next iteration
|
|
|
|
#### Scenario: Exception during sync cycle
|
|
|
|
- **WHEN** an exception occurs during sync execution (not OperationCanceledException)
|
|
- **THEN** the exception SHALL be caught and logged with LogError
|
|
- **THEN** the service SHALL continue to the next iteration
|
|
- **THEN** the current scope SHALL still be disposed properly
|
|
- **THEN** the service SHALL NOT crash or stop unexpectedly
|
|
|
|
#### Scenario: Graceful shutdown request
|
|
|
|
- **WHEN** the host signals shutdown by canceling the stoppingToken
|
|
- **THEN** any running Task.Delay SHALL throw OperationCanceledException
|
|
- **THEN** the while loop SHALL exit on the IsCancellationRequested check
|
|
- **THEN** the ExecuteAsync method SHALL complete normally
|
|
- **THEN** any in-progress sync operations SHALL receive the cancellation and complete or cancel
|
|
|
|
### Requirement: Parallel fetch isolation with scoped resources
|
|
|
|
The system SHALL ensure complete isolation between parallel sync operations using scoped resources and unique identifiers.
|
|
|
|
#### Inputs
|
|
|
|
- List of `DataUpdateTask` objects to execute in parallel
|
|
- `MaxDegreeOfParallelism` configuration value
|
|
- `CancellationToken` for coordinated cancellation
|
|
|
|
#### Outputs
|
|
|
|
- Concurrent execution of sync operations with no resource conflicts
|
|
- Unique staging tables per operation that do not collide
|
|
- Independent database connections per operation
|
|
|
|
#### Business Rules
|
|
|
|
- `Parallel.ForEachAsync` MUST be used with `ParallelOptions.CancellationToken` set
|
|
- Each parallel task MUST create its own `IServiceScope` inside the parallel delegate
|
|
- Database connections MUST NOT be shared across parallel operations
|
|
- Staging table names MUST include a unique `OperationId` suffix (GUID or sequential ID)
|
|
- Format: `#Staging{TableName}_{OperationId}` and `#{TableName}_{OperationId}`
|
|
- Each parallel operation MUST resolve its own instances of all scoped services
|
|
- No `ConcurrentDictionary`, shared counters, or other shared mutable state SHALL exist between operations
|
|
- Total record counts SHALL be accumulated via return values, not shared state
|
|
|
|
#### Scenario: Parallel sync with isolated scopes
|
|
|
|
- **WHEN** multiple DataUpdateTasks are executed via Parallel.ForEachAsync
|
|
- **THEN** each task SHALL execute the async delegate independently
|
|
- **THEN** each delegate SHALL create a new IServiceScope using CreateAsyncScope
|
|
- **THEN** ITableSyncOperation SHALL be resolved from each scope independently
|
|
- **THEN** each operation SHALL use its own database connection from the scope
|
|
- **THEN** staging tables SHALL use unique OperationId suffixes preventing name collisions
|
|
- **THEN** completion of one operation SHALL NOT affect the execution of others
|
|
|
|
#### Scenario: Parallel cancellation propagation
|
|
|
|
- **WHEN** cancellation is requested during Parallel.ForEachAsync execution
|
|
- **THEN** the CancellationToken SHALL propagate to all running parallel operations
|
|
- **THEN** Parallel.ForEachAsync SHALL stop starting new operations
|
|
- **THEN** running operations SHALL receive the token in their async methods
|
|
- **THEN** each operation SHALL check the token and exit gracefully
|
|
- **THEN** incomplete operations SHALL mark their DataUpdate records as failed
|
|
|
|
#### Scenario: Staging table uniqueness verification
|
|
|
|
- **WHEN** two sync operations for the same table run in parallel
|
|
- **THEN** each operation SHALL generate a unique OperationId as GUID
|
|
- **THEN** operation A SHALL create staging table with GuidA suffix
|
|
- **THEN** operation B SHALL create staging table with GuidB suffix
|
|
- **THEN** no SQL errors SHALL occur from table name conflicts
|
|
- **THEN** each operation cleanup SHALL only drop its own staging tables
|
|
|
|
### Requirement: Structured logging context
|
|
|
|
The system SHALL use ILogger.BeginScope to attach contextual information to all log entries during sync operations.
|
|
|
|
#### Inputs
|
|
|
|
- `ILogger<T>` injected into sync operation classes
|
|
- TableName, UpdateType, OperationId values from current operation
|
|
|
|
#### Outputs
|
|
|
|
- All log entries within the scope contain the contextual properties
|
|
- Log aggregation systems can filter and group by table, type, or operation
|
|
|
|
#### Business Rules
|
|
|
|
- Each sync operation MUST call `_logger.BeginScope(...)` at the start
|
|
- The scope MUST include at minimum: TableName, UpdateType, OperationId
|
|
- The scope MUST be disposed using `using` statement when operation completes
|
|
- Nested scopes for batches SHALL preserve parent scope properties
|
|
- LogInformation, LogWarning, LogError calls within the scope SHALL include the context automatically
|
|
|
|
#### Scenario: Log scope creation and usage
|
|
|
|
- **WHEN** a TableSyncOperation begins execution
|
|
- **THEN** the operation SHALL create a logging scope with TableName, UpdateType, OperationId
|
|
- **THEN** all log calls within ExecuteAsync SHALL include these properties
|
|
- **THEN** when the operation completes the scope SHALL be disposed
|
|
- **THEN** subsequent operations SHALL have their own independent scopes
|
|
|
|
## Migration Notes
|
|
|
|
| Legacy Pattern | New Pattern | Rationale |
|
|
|----------------|-------------|-----------|
|
|
| Static `UpdateProcessor` methods | Scoped services resolved per operation | Proper DI lifecycle, testability |
|
|
| Shared instance state | Return values and scoped state only | Thread safety in parallel scenarios |
|
|
| `Console.WriteLine` logging | `ILogger<T>` with `BeginScope` | Structured logging, context propagation |
|
|
| Global temp tables `##table` | Local temp tables `#table_{id}` | Session-scoped isolation for parallelism |
|