# Data Sync Specification - Implementation Additions ## Purpose This specification extends the base data-sync spec with additional implementation-focused requirements for the BackgroundService pattern and parallel fetch isolation. ## ADDED Requirements ### Requirement: Background service implementation pattern The system SHALL implement the data synchronization service following .NET BackgroundService best practices for hosted service lifecycle management. #### Inputs - `IServiceScopeFactory` for creating scoped service instances - `IOptions` for configuration access - `ILogger` for structured logging - `CancellationToken` from `ExecuteAsync` stoppingToken parameter #### Outputs - Continuously running background task that checks schedules and executes syncs - Proper cleanup on shutdown with all resources disposed - Logging scope context for all operations #### Business Rules - The service MUST implement `BackgroundService.ExecuteAsync(CancellationToken)` - The main loop MUST use `Task.Delay(checkInterval, stoppingToken)` between cycles - Each sync cycle MUST create a new `IServiceScope` via `IServiceScopeFactory.CreateAsyncScope()` - All scoped services MUST be resolved from the current scope, not from root provider - The scope MUST be disposed using `await using` pattern after each cycle - Exception handling MUST catch and log errors without crashing the service - `OperationCanceledException` MUST be caught and result in graceful loop exit when `stoppingToken.IsCancellationRequested` - The service MUST NOT use static state or shared mutable collections #### Scenario: Normal sync cycle execution - **WHEN** the BackgroundService enters ExecuteAsync - **THEN** the service SHALL call CloseOpenUpdateEntriesAsync to recover from prior crashes - **THEN** the service SHALL enter a while loop checking `!stoppingToken.IsCancellationRequested` - **THEN** each iteration SHALL create a new IServiceScope - **THEN** the ISyncOrchestrator SHALL be resolved from the scope - **THEN** ExecutePendingSyncsAsync SHALL be called with the stoppingToken - **THEN** the scope SHALL be disposed after the call completes - **THEN** Task.Delay SHALL pause before the next iteration #### Scenario: Exception during sync cycle - **WHEN** an exception occurs during sync execution (not OperationCanceledException) - **THEN** the exception SHALL be caught and logged with LogError - **THEN** the service SHALL continue to the next iteration - **THEN** the current scope SHALL still be disposed properly - **THEN** the service SHALL NOT crash or stop unexpectedly #### Scenario: Graceful shutdown request - **WHEN** the host signals shutdown by canceling the stoppingToken - **THEN** any running Task.Delay SHALL throw OperationCanceledException - **THEN** the while loop SHALL exit on the IsCancellationRequested check - **THEN** the ExecuteAsync method SHALL complete normally - **THEN** any in-progress sync operations SHALL receive the cancellation and complete or cancel ### Requirement: Parallel fetch isolation with scoped resources The system SHALL ensure complete isolation between parallel sync operations using scoped resources and unique identifiers. #### Inputs - List of `DataUpdateTask` objects to execute in parallel - `MaxDegreeOfParallelism` configuration value - `CancellationToken` for coordinated cancellation #### Outputs - Concurrent execution of sync operations with no resource conflicts - Unique staging tables per operation that do not collide - Independent database connections per operation #### Business Rules - `Parallel.ForEachAsync` MUST be used with `ParallelOptions.CancellationToken` set - Each parallel task MUST create its own `IServiceScope` inside the parallel delegate - Database connections MUST NOT be shared across parallel operations - Staging table names MUST include a unique `OperationId` suffix (GUID or sequential ID) - Format: `#Staging{TableName}_{OperationId}` and `#{TableName}_{OperationId}` - Each parallel operation MUST resolve its own instances of all scoped services - No `ConcurrentDictionary`, shared counters, or other shared mutable state SHALL exist between operations - Total record counts SHALL be accumulated via return values, not shared state #### Scenario: Parallel sync with isolated scopes - **WHEN** multiple DataUpdateTasks are executed via Parallel.ForEachAsync - **THEN** each task SHALL execute the async delegate independently - **THEN** each delegate SHALL create a new IServiceScope using CreateAsyncScope - **THEN** ITableSyncOperation SHALL be resolved from each scope independently - **THEN** each operation SHALL use its own database connection from the scope - **THEN** staging tables SHALL use unique OperationId suffixes preventing name collisions - **THEN** completion of one operation SHALL NOT affect the execution of others #### Scenario: Parallel cancellation propagation - **WHEN** cancellation is requested during Parallel.ForEachAsync execution - **THEN** the CancellationToken SHALL propagate to all running parallel operations - **THEN** Parallel.ForEachAsync SHALL stop starting new operations - **THEN** running operations SHALL receive the token in their async methods - **THEN** each operation SHALL check the token and exit gracefully - **THEN** incomplete operations SHALL mark their DataUpdate records as failed #### Scenario: Staging table uniqueness verification - **WHEN** two sync operations for the same table run in parallel - **THEN** each operation SHALL generate a unique OperationId as GUID - **THEN** operation A SHALL create staging table with GuidA suffix - **THEN** operation B SHALL create staging table with GuidB suffix - **THEN** no SQL errors SHALL occur from table name conflicts - **THEN** each operation cleanup SHALL only drop its own staging tables ### Requirement: Structured logging context The system SHALL use ILogger.BeginScope to attach contextual information to all log entries during sync operations. #### Inputs - `ILogger` injected into sync operation classes - TableName, UpdateType, OperationId values from current operation #### Outputs - All log entries within the scope contain the contextual properties - Log aggregation systems can filter and group by table, type, or operation #### Business Rules - Each sync operation MUST call `_logger.BeginScope(...)` at the start - The scope MUST include at minimum: TableName, UpdateType, OperationId - The scope MUST be disposed using `using` statement when operation completes - Nested scopes for batches SHALL preserve parent scope properties - LogInformation, LogWarning, LogError calls within the scope SHALL include the context automatically #### Scenario: Log scope creation and usage - **WHEN** a TableSyncOperation begins execution - **THEN** the operation SHALL create a logging scope with TableName, UpdateType, OperationId - **THEN** all log calls within ExecuteAsync SHALL include these properties - **THEN** when the operation completes the scope SHALL be disposed - **THEN** subsequent operations SHALL have their own independent scopes ## Migration Notes | Legacy Pattern | New Pattern | Rationale | |----------------|-------------|-----------| | Static `UpdateProcessor` methods | Scoped services resolved per operation | Proper DI lifecycle, testability | | Shared instance state | Return values and scoped state only | Thread safety in parallel scenarios | | `Console.WriteLine` logging | `ILogger` with `BeginScope` | Structured logging, context propagation | | Global temp tables `##table` | Local temp tables `#table_{id}` | Session-scoped isolation for parallelism |