Joseph Doherty
ff487aa99c
Detailed step-by-step implementation tasks: 1. Create Pipeline DTOs in Core 2. Add Pipeline Routes to ApiRoutes 3. Add Repository Methods 4. Implement Repository Methods 5. Create PipelineController 6. Add Missing Factory Methods 7. Create Client API Service 8. Create SqlQueryModal Component 9. Create PipelineScheduleSection Component 10. Create PipelineViewer Page 11. Register Services 12. Add Navigation Link
1328 lines
45 KiB
Markdown
1328 lines
45 KiB
Markdown
# ETL Pipeline Viewer Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Implement the ETL Pipeline Viewer Blazor component with API backend.
|
|
|
|
**Architecture:** API-first approach - build DTOs and API endpoints first, then client components.
|
|
|
|
**Tech Stack:** .NET 10, ASP.NET Core API, Blazor WebAssembly, Radzen, Dapper
|
|
|
|
---
|
|
|
|
## Task 1: Create Pipeline DTOs in Core
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Core/ApiContracts/Pipelines/PipelineListResponse.cs`
|
|
- Create: `src/JdeScoping.Core/ApiContracts/Pipelines/PipelineConfigDto.cs`
|
|
- Create: `src/JdeScoping.Core/ApiContracts/Pipelines/PipelineStatusDto.cs`
|
|
- Create: `src/JdeScoping.Core/ApiContracts/Pipelines/PipelineExecutionDto.cs`
|
|
|
|
**Step 1: Create the Pipelines directory**
|
|
|
|
```bash
|
|
mkdir -p src/JdeScoping.Core/ApiContracts/Pipelines
|
|
```
|
|
|
|
**Step 2: Create PipelineListResponse.cs**
|
|
|
|
```csharp
|
|
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
|
|
|
/// <summary>
|
|
/// Response containing list of available pipeline names.
|
|
/// </summary>
|
|
public record PipelineListResponse(List<string> PipelineNames);
|
|
```
|
|
|
|
**Step 3: Create PipelineConfigDto.cs**
|
|
|
|
```csharp
|
|
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
|
|
|
/// <summary>
|
|
/// Pipeline configuration DTO for display.
|
|
/// </summary>
|
|
public record PipelineConfigDto(
|
|
string Name,
|
|
PipelineSourceDto Source,
|
|
PipelineDestinationDto Destination,
|
|
PipelineSchedulesDto Schedules,
|
|
int PreScriptCount,
|
|
int PostScriptCount,
|
|
List<string>? PreScripts,
|
|
List<string>? PostScripts);
|
|
|
|
public record PipelineSourceDto(
|
|
string Connection,
|
|
string? QueryPreview,
|
|
string? MassQueryPreview,
|
|
string? Query,
|
|
string? MassQuery,
|
|
List<PipelineParameterDto> Parameters);
|
|
|
|
public record PipelineParameterDto(
|
|
string Name,
|
|
string? Format,
|
|
string Source);
|
|
|
|
public record PipelineDestinationDto(
|
|
string Table,
|
|
string OperationType,
|
|
List<string>? MatchColumns,
|
|
List<string>? ExcludeFromUpdate);
|
|
|
|
public record PipelineSchedulesDto(
|
|
PipelineScheduleDto Mass,
|
|
PipelineScheduleDto Daily,
|
|
PipelineScheduleDto Hourly);
|
|
|
|
public record PipelineScheduleDto(
|
|
bool Enabled,
|
|
int IntervalMinutes,
|
|
bool PrePurge,
|
|
bool ReIndex,
|
|
bool IntervalIsOverride,
|
|
bool PrePurgeIsOverride,
|
|
bool ReIndexIsOverride);
|
|
```
|
|
|
|
**Step 4: Create PipelineStatusDto.cs**
|
|
|
|
```csharp
|
|
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
|
|
|
using JdeScoping.Core.Models.Enums;
|
|
|
|
/// <summary>
|
|
/// Pipeline schedule status for each update type.
|
|
/// </summary>
|
|
public record PipelineStatusResponse(List<PipelineScheduleStatusDto> Statuses);
|
|
|
|
public record PipelineScheduleStatusDto(
|
|
UpdateTypes ScheduleType,
|
|
DateTime? LastRun,
|
|
bool LastRunWasSuccessful,
|
|
DateTime? LastSuccessfulRun,
|
|
DateTime? NextRequiredRun,
|
|
bool IsOverdue,
|
|
int IntervalMinutes);
|
|
```
|
|
|
|
**Step 5: Create PipelineExecutionDto.cs**
|
|
|
|
```csharp
|
|
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
|
|
|
using JdeScoping.Core.Models.Enums;
|
|
|
|
/// <summary>
|
|
/// Pipeline execution history.
|
|
/// </summary>
|
|
public record PipelineExecutionsResponse(List<PipelineExecutionDto> Executions);
|
|
|
|
public record PipelineExecutionDto(
|
|
UpdateTypes ScheduleType,
|
|
DateTime StartTime,
|
|
DateTime? EndTime,
|
|
TimeSpan? Duration,
|
|
long RecordCount,
|
|
bool WasSuccessful);
|
|
```
|
|
|
|
**Step 6: Verify build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Core
|
|
```
|
|
|
|
**Step 7: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Core/ApiContracts/Pipelines/
|
|
git commit -m "feat(core): add pipeline viewer DTOs"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Add Pipeline Routes to ApiRoutes
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.Core/ApiContracts/ApiRoutes.cs`
|
|
|
|
**Step 1: Add Pipelines routes class after FileIO class**
|
|
|
|
Add at the end of `ApiRoutes.cs`, before the closing brace:
|
|
|
|
```csharp
|
|
/// <summary>
|
|
/// Routes for pipeline configuration API endpoints.
|
|
/// </summary>
|
|
public static class Pipelines
|
|
{
|
|
/// <summary>Base route for pipeline endpoints.</summary>
|
|
public const string Base = "api/pipelines";
|
|
|
|
/// <summary>Route template for getting a pipeline by name.</summary>
|
|
public const string ByName = "{name}";
|
|
|
|
/// <summary>Route template for getting pipeline status.</summary>
|
|
public const string Status = "{name}/status";
|
|
|
|
/// <summary>Route template for getting pipeline executions.</summary>
|
|
public const string Executions = "{name}/executions";
|
|
|
|
/// <summary>Builds the route to get a specific pipeline config.</summary>
|
|
public static string GetByName(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}";
|
|
|
|
/// <summary>Builds the route to get pipeline status.</summary>
|
|
public static string GetStatus(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}/status";
|
|
|
|
/// <summary>Builds the route to get pipeline executions.</summary>
|
|
public static string GetExecutions(string name, int count = 10) =>
|
|
$"api/pipelines/{Uri.EscapeDataString(name)}/executions?count={count}";
|
|
}
|
|
```
|
|
|
|
**Step 2: Verify build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Core
|
|
```
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Core/ApiContracts/ApiRoutes.cs
|
|
git commit -m "feat(core): add pipeline API routes"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Add Repository Methods to IDataUpdateRepository
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.DataSync/Contracts/IDataUpdateRepository.cs`
|
|
|
|
**Step 1: Add new method signatures**
|
|
|
|
Add these methods to the interface:
|
|
|
|
```csharp
|
|
/// <summary>
|
|
/// Gets the last N execution records for a specific table.
|
|
/// </summary>
|
|
/// <param name="tableName">The table name to filter by.</param>
|
|
/// <param name="updateType">Optional update type filter. If null, returns all types.</param>
|
|
/// <param name="count">Maximum records to return (total, not per type).</param>
|
|
/// <param name="cancellationToken">Cancellation token.</param>
|
|
/// <returns>List of DataUpdate records ordered by StartDt descending.</returns>
|
|
Task<List<DataUpdate>> GetRecentUpdatesAsync(
|
|
string tableName,
|
|
UpdateTypes? updateType = null,
|
|
int count = 30,
|
|
CancellationToken cancellationToken = default);
|
|
|
|
/// <summary>
|
|
/// Gets the last run (successful or not) for each update type for a table.
|
|
/// </summary>
|
|
/// <param name="tableName">The table name.</param>
|
|
/// <param name="cancellationToken">Cancellation token.</param>
|
|
/// <returns>Dictionary keyed by UpdateType.</returns>
|
|
Task<Dictionary<UpdateTypes, DataUpdate>> GetLastRunsAsync(
|
|
string tableName,
|
|
CancellationToken cancellationToken = default);
|
|
```
|
|
|
|
**Step 2: Verify build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.DataSync
|
|
```
|
|
|
|
Expected: Build fails (not implemented yet)
|
|
|
|
**Step 3: Commit interface changes**
|
|
|
|
```bash
|
|
git add src/JdeScoping.DataSync/Contracts/IDataUpdateRepository.cs
|
|
git commit -m "feat(datasync): add repository methods for pipeline viewer"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Implement Repository Methods
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.DataSync/Services/DataUpdateRepository.cs`
|
|
|
|
**Step 1: Implement GetRecentUpdatesAsync**
|
|
|
|
Add this method to `DataUpdateRepository`:
|
|
|
|
```csharp
|
|
/// <inheritdoc/>
|
|
public async Task<List<DataUpdate>> GetRecentUpdatesAsync(
|
|
string tableName,
|
|
UpdateTypes? updateType = null,
|
|
int count = 30,
|
|
CancellationToken cancellationToken = default)
|
|
{
|
|
var sql = updateType.HasValue
|
|
? @"SELECT TOP (@count) du.Id, du.SourceSystem, du.SourceData, du.TableName,
|
|
du.StartDt, du.EndDt, du.UpdateType, du.WasSuccessful, du.NumberRecords
|
|
FROM dbo.DataUpdate du
|
|
WHERE du.TableName = @tableName AND du.UpdateType = @updateType
|
|
ORDER BY du.StartDt DESC"
|
|
: @"SELECT TOP (@count) du.Id, du.SourceSystem, du.SourceData, du.TableName,
|
|
du.StartDt, du.EndDt, du.UpdateType, du.WasSuccessful, du.NumberRecords
|
|
FROM dbo.DataUpdate du
|
|
WHERE du.TableName = @tableName
|
|
ORDER BY du.StartDt DESC";
|
|
|
|
await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(cancellationToken);
|
|
var results = await connection.QueryAsync<DataUpdate>(
|
|
sql,
|
|
new { tableName, updateType = updateType.HasValue ? (int)updateType.Value : 0, count },
|
|
commandTimeout: 30);
|
|
|
|
return results.ToList();
|
|
}
|
|
```
|
|
|
|
**Step 2: Implement GetLastRunsAsync**
|
|
|
|
Add this method to `DataUpdateRepository`:
|
|
|
|
```csharp
|
|
/// <inheritdoc/>
|
|
public async Task<Dictionary<UpdateTypes, DataUpdate>> GetLastRunsAsync(
|
|
string tableName,
|
|
CancellationToken cancellationToken = default)
|
|
{
|
|
const string sql = @"
|
|
WITH LastRuns AS (
|
|
SELECT du.*,
|
|
ROW_NUMBER() OVER (PARTITION BY du.UpdateType ORDER BY du.StartDt DESC) AS RN
|
|
FROM dbo.DataUpdate du
|
|
WHERE du.TableName = @tableName
|
|
)
|
|
SELECT Id, SourceSystem, SourceData, TableName, StartDt, EndDt, UpdateType, WasSuccessful, NumberRecords
|
|
FROM LastRuns
|
|
WHERE RN = 1";
|
|
|
|
await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(cancellationToken);
|
|
var results = await connection.QueryAsync<DataUpdate>(sql, new { tableName }, commandTimeout: 30);
|
|
|
|
return results.ToDictionary(du => du.UpdateType, du => du);
|
|
}
|
|
```
|
|
|
|
**Step 3: Verify build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.DataSync
|
|
```
|
|
|
|
**Step 4: Run existing tests**
|
|
|
|
```bash
|
|
dotnet test tests/JdeScoping.DataSync.Tests --filter "DataUpdateRepository" --no-build
|
|
```
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.DataSync/Services/DataUpdateRepository.cs
|
|
git commit -m "feat(datasync): implement GetRecentUpdatesAsync and GetLastRunsAsync"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 5: Create PipelineController
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Api/Controllers/PipelineController.cs`
|
|
|
|
**Step 1: Create the controller**
|
|
|
|
```csharp
|
|
using JdeScoping.Core.ApiContracts;
|
|
using JdeScoping.Core.ApiContracts.Pipelines;
|
|
using JdeScoping.Core.Models.Enums;
|
|
using JdeScoping.DataSync.Contracts;
|
|
using Microsoft.AspNetCore.Authorization;
|
|
using Microsoft.AspNetCore.Mvc;
|
|
|
|
namespace JdeScoping.Api.Controllers;
|
|
|
|
/// <summary>
|
|
/// API endpoints for pipeline configuration and status.
|
|
/// </summary>
|
|
[ApiController]
|
|
[Route(ApiRoutes.Pipelines.Base)]
|
|
[Authorize]
|
|
public class PipelineController : ControllerBase
|
|
{
|
|
private readonly IEtlPipelineFactory _pipelineFactory;
|
|
private readonly IDataUpdateRepository _dataUpdateRepository;
|
|
|
|
public PipelineController(
|
|
IEtlPipelineFactory pipelineFactory,
|
|
IDataUpdateRepository dataUpdateRepository)
|
|
{
|
|
_pipelineFactory = pipelineFactory;
|
|
_dataUpdateRepository = dataUpdateRepository;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Gets list of all available pipeline names.
|
|
/// </summary>
|
|
[HttpGet]
|
|
public ActionResult<PipelineListResponse> GetPipelineNames()
|
|
{
|
|
var names = _pipelineFactory.GetAvailableTables()
|
|
.OrderBy(n => n)
|
|
.ToList();
|
|
return Ok(new PipelineListResponse(names));
|
|
}
|
|
|
|
/// <summary>
|
|
/// Gets configuration for a specific pipeline.
|
|
/// </summary>
|
|
[HttpGet(ApiRoutes.Pipelines.ByName)]
|
|
public ActionResult<PipelineConfigDto> GetPipeline(string name)
|
|
{
|
|
var config = _pipelineFactory.GetPipelineConfig(name);
|
|
if (config is null)
|
|
return NotFound();
|
|
|
|
var defaults = _pipelineFactory.GetScheduleDefaults();
|
|
var dto = MapToDto(name, config, defaults);
|
|
return Ok(dto);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Gets schedule status for a pipeline.
|
|
/// </summary>
|
|
[HttpGet(ApiRoutes.Pipelines.Status)]
|
|
public async Task<ActionResult<PipelineStatusResponse>> GetStatus(
|
|
string name,
|
|
CancellationToken cancellationToken)
|
|
{
|
|
var config = _pipelineFactory.GetPipelineConfig(name);
|
|
if (config is null)
|
|
return NotFound();
|
|
|
|
var tableName = config.Destination.Table;
|
|
var lastRuns = await _dataUpdateRepository.GetLastRunsAsync(tableName, cancellationToken);
|
|
var lastSuccessful = await _dataUpdateRepository.GetLastDataUpdatesAsync(cancellationToken);
|
|
var defaults = _pipelineFactory.GetScheduleDefaults();
|
|
|
|
var statuses = new List<PipelineScheduleStatusDto>();
|
|
foreach (var updateType in new[] { UpdateTypes.Mass, UpdateTypes.Daily, UpdateTypes.Hourly })
|
|
{
|
|
var scheduleConfig = GetScheduleConfig(config, updateType);
|
|
var interval = GetEffectiveInterval(scheduleConfig, defaults, updateType);
|
|
|
|
lastRuns.TryGetValue(updateType, out var lastRun);
|
|
var successKey = $"{tableName}_{(int)updateType}";
|
|
lastSuccessful.TryGetValue(successKey, out var lastSuccess);
|
|
|
|
var nextRequired = lastSuccess?.EndDt.AddMinutes(interval);
|
|
var isOverdue = DataUpdateRepository.IsOverdue(
|
|
lastSuccess?.EndDt, tableName, updateType, null);
|
|
|
|
statuses.Add(new PipelineScheduleStatusDto(
|
|
updateType,
|
|
lastRun?.StartDt,
|
|
lastRun?.WasSuccessful ?? false,
|
|
lastSuccess?.EndDt,
|
|
nextRequired,
|
|
isOverdue,
|
|
interval));
|
|
}
|
|
|
|
return Ok(new PipelineStatusResponse(statuses));
|
|
}
|
|
|
|
/// <summary>
|
|
/// Gets recent execution history for a pipeline.
|
|
/// </summary>
|
|
[HttpGet(ApiRoutes.Pipelines.Executions)]
|
|
public async Task<ActionResult<PipelineExecutionsResponse>> GetExecutions(
|
|
string name,
|
|
[FromQuery] int count = 30,
|
|
CancellationToken cancellationToken = default)
|
|
{
|
|
var config = _pipelineFactory.GetPipelineConfig(name);
|
|
if (config is null)
|
|
return NotFound();
|
|
|
|
var tableName = config.Destination.Table;
|
|
var updates = await _dataUpdateRepository.GetRecentUpdatesAsync(
|
|
tableName, null, count, cancellationToken);
|
|
|
|
var executions = updates.Select(u => new PipelineExecutionDto(
|
|
u.UpdateType,
|
|
u.StartDt,
|
|
u.EndDt == default ? null : u.EndDt,
|
|
u.EndDt == default ? null : u.EndDt - u.StartDt,
|
|
u.NumberRecords,
|
|
u.WasSuccessful
|
|
)).ToList();
|
|
|
|
return Ok(new PipelineExecutionsResponse(executions));
|
|
}
|
|
|
|
private PipelineConfigDto MapToDto(
|
|
string name,
|
|
DataSync.Configuration.PipelineConfig config,
|
|
DataSync.Configuration.ScheduleDefaults defaults)
|
|
{
|
|
var source = new PipelineSourceDto(
|
|
config.Source.Connection,
|
|
Truncate(config.Source.Query),
|
|
Truncate(config.Source.MassQuery),
|
|
config.Source.Query,
|
|
config.Source.MassQuery,
|
|
config.Source.Parameters?.Select(p => new PipelineParameterDto(
|
|
p.Key, p.Value.Format, p.Value.Source)).ToList() ?? []);
|
|
|
|
var matchCols = config.Destination.MatchColumns?.ToList();
|
|
var destination = new PipelineDestinationDto(
|
|
config.Destination.Table,
|
|
matchCols?.Count > 0 ? "BulkMerge" : "BulkImport",
|
|
matchCols,
|
|
config.Destination.ExcludeFromUpdate?.ToList());
|
|
|
|
var schedules = new PipelineSchedulesDto(
|
|
MapSchedule(config.Schedules?.Mass, defaults.Mass),
|
|
MapSchedule(config.Schedules?.Daily, defaults.Daily),
|
|
MapSchedule(config.Schedules?.Hourly, defaults.Hourly));
|
|
|
|
return new PipelineConfigDto(
|
|
name,
|
|
source,
|
|
destination,
|
|
schedules,
|
|
config.PreScripts?.Count ?? 0,
|
|
config.PostScripts?.Count ?? 0,
|
|
config.PreScripts,
|
|
config.PostScripts);
|
|
}
|
|
|
|
private static PipelineScheduleDto MapSchedule(
|
|
DataSync.Configuration.ScheduleConfig? config,
|
|
DataSync.Configuration.ScheduleConfig defaults)
|
|
{
|
|
return new PipelineScheduleDto(
|
|
config?.Enabled ?? defaults.Enabled ?? true,
|
|
config?.IntervalMinutes ?? defaults.IntervalMinutes ?? 60,
|
|
config?.PrePurge ?? defaults.PrePurge ?? false,
|
|
config?.ReIndex ?? defaults.ReIndex ?? false,
|
|
config?.IntervalMinutes.HasValue == true && config.IntervalMinutes != defaults.IntervalMinutes,
|
|
config?.PrePurge.HasValue == true && config.PrePurge != defaults.PrePurge,
|
|
config?.ReIndex.HasValue == true && config.ReIndex != defaults.ReIndex);
|
|
}
|
|
|
|
private static DataSync.Configuration.ScheduleConfig? GetScheduleConfig(
|
|
DataSync.Configuration.PipelineConfig config,
|
|
UpdateTypes updateType) => updateType switch
|
|
{
|
|
UpdateTypes.Mass => config.Schedules?.Mass,
|
|
UpdateTypes.Daily => config.Schedules?.Daily,
|
|
UpdateTypes.Hourly => config.Schedules?.Hourly,
|
|
_ => null
|
|
};
|
|
|
|
private static int GetEffectiveInterval(
|
|
DataSync.Configuration.ScheduleConfig? config,
|
|
DataSync.Configuration.ScheduleDefaults defaults,
|
|
UpdateTypes updateType)
|
|
{
|
|
if (config?.IntervalMinutes.HasValue == true)
|
|
return config.IntervalMinutes.Value;
|
|
|
|
return updateType switch
|
|
{
|
|
UpdateTypes.Mass => defaults.Mass?.IntervalMinutes ?? 10080,
|
|
UpdateTypes.Daily => defaults.Daily?.IntervalMinutes ?? 1440,
|
|
UpdateTypes.Hourly => defaults.Hourly?.IntervalMinutes ?? 60,
|
|
_ => 60
|
|
};
|
|
}
|
|
|
|
private static string? Truncate(string? value, int maxLength = 100) =>
|
|
value is null ? null :
|
|
value.Length <= maxLength ? value : value[..maxLength] + "...";
|
|
}
|
|
```
|
|
|
|
**Step 2: Verify IEtlPipelineFactory has needed methods**
|
|
|
|
Check if `GetPipelineConfig` and `GetScheduleDefaults` exist. If not, we need to add them.
|
|
|
|
**Step 3: Build and fix any issues**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Api
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Api/Controllers/PipelineController.cs
|
|
git commit -m "feat(api): add PipelineController for pipeline viewer"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 6: Add Missing Factory Methods (if needed)
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.DataSync/Contracts/IEtlPipelineFactory.cs`
|
|
- Modify: `src/JdeScoping.DataSync/Services/EtlPipelineFactory.cs`
|
|
|
|
**Step 1: Check if methods exist**
|
|
|
|
Read `IEtlPipelineFactory.cs` and check for `GetPipelineConfig` and `GetScheduleDefaults`.
|
|
|
|
**Step 2: If missing, add to interface**
|
|
|
|
```csharp
|
|
/// <summary>
|
|
/// Gets the configuration for a specific pipeline.
|
|
/// </summary>
|
|
PipelineConfig? GetPipelineConfig(string tableName);
|
|
|
|
/// <summary>
|
|
/// Gets the schedule defaults.
|
|
/// </summary>
|
|
ScheduleDefaults GetScheduleDefaults();
|
|
```
|
|
|
|
**Step 3: Implement in EtlPipelineFactory**
|
|
|
|
```csharp
|
|
public PipelineConfig? GetPipelineConfig(string tableName)
|
|
{
|
|
return _config.Pipelines.TryGetValue(tableName, out var config) ? config : null;
|
|
}
|
|
|
|
public ScheduleDefaults GetScheduleDefaults()
|
|
{
|
|
return _config.ScheduleDefaults ?? new ScheduleDefaults(
|
|
new ScheduleConfig { IntervalMinutes = 10080, PrePurge = true, ReIndex = true },
|
|
new ScheduleConfig { IntervalMinutes = 1440 },
|
|
new ScheduleConfig { IntervalMinutes = 60 });
|
|
}
|
|
```
|
|
|
|
**Step 4: Build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.DataSync && dotnet build src/JdeScoping.Api
|
|
```
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.DataSync/
|
|
git commit -m "feat(datasync): add GetPipelineConfig and GetScheduleDefaults to factory"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 7: Create Client API Service
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Client/Services/IPipelineApiClient.cs`
|
|
- Create: `src/JdeScoping.Client/Services/PipelineApiClient.cs`
|
|
|
|
**Step 1: Create IPipelineApiClient.cs**
|
|
|
|
```csharp
|
|
using JdeScoping.Core.ApiContracts.Pipelines;
|
|
using JdeScoping.Core.ApiContracts.Results;
|
|
|
|
namespace JdeScoping.Client.Services;
|
|
|
|
/// <summary>
|
|
/// Client for pipeline configuration API.
|
|
/// </summary>
|
|
public interface IPipelineApiClient
|
|
{
|
|
Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default);
|
|
Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default);
|
|
Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default);
|
|
Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 30, CancellationToken ct = default);
|
|
}
|
|
```
|
|
|
|
**Step 2: Create PipelineApiClient.cs**
|
|
|
|
```csharp
|
|
using JdeScoping.Core.ApiContracts;
|
|
using JdeScoping.Core.ApiContracts.Pipelines;
|
|
using JdeScoping.Core.ApiContracts.Results;
|
|
|
|
namespace JdeScoping.Client.Services;
|
|
|
|
/// <summary>
|
|
/// Client implementation for pipeline configuration API.
|
|
/// </summary>
|
|
public class PipelineApiClient : ApiClientBase, IPipelineApiClient
|
|
{
|
|
public PipelineApiClient(HttpClient httpClient) : base(httpClient) { }
|
|
|
|
public Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default)
|
|
=> GetAsync<PipelineListResponse>(ApiRoutes.Pipelines.Base, ct);
|
|
|
|
public Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default)
|
|
=> GetAsync<PipelineConfigDto>(ApiRoutes.Pipelines.GetByName(name), ct);
|
|
|
|
public Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default)
|
|
=> GetAsync<PipelineStatusResponse>(ApiRoutes.Pipelines.GetStatus(name), ct);
|
|
|
|
public Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 30, CancellationToken ct = default)
|
|
=> GetAsync<PipelineExecutionsResponse>(ApiRoutes.Pipelines.GetExecutions(name, count), ct);
|
|
}
|
|
```
|
|
|
|
**Step 3: Register in DI (Program.cs)**
|
|
|
|
Add to client DI registration:
|
|
|
|
```csharp
|
|
builder.Services.AddScoped<IPipelineApiClient, PipelineApiClient>();
|
|
```
|
|
|
|
**Step 4: Build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Client
|
|
```
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/Services/IPipelineApiClient.cs src/JdeScoping.Client/Services/PipelineApiClient.cs
|
|
git commit -m "feat(client): add PipelineApiClient"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 8: Create SqlQueryModal Component
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Client/Components/Admin/SqlQueryModal.razor`
|
|
|
|
**Step 1: Create Admin directory**
|
|
|
|
```bash
|
|
mkdir -p src/JdeScoping.Client/Components/Admin
|
|
```
|
|
|
|
**Step 2: Create SqlQueryModal.razor**
|
|
|
|
```razor
|
|
@namespace JdeScoping.Client.Components.Admin
|
|
|
|
<RadzenDialog Visible="@Visible" Style="width: 80vw; max-width: 1200px;" ShowClose="true" CloseOnEscapeKey="true">
|
|
<TitleContent>
|
|
<RadzenText TextStyle="TextStyle.H5">@Title</RadzenText>
|
|
</TitleContent>
|
|
<ChildContent>
|
|
<div class="sql-container" style="max-height: 60vh; overflow: auto; background: #f5f5f5; padding: 1rem; border-radius: 4px;">
|
|
<pre style="margin: 0; white-space: pre-wrap; word-break: break-word; font-family: 'Consolas', 'Monaco', monospace; font-size: 0.875rem;">@FormattedSql</pre>
|
|
</div>
|
|
</ChildContent>
|
|
<FooterContent>
|
|
<RadzenButton Text="Copy to Clipboard" Icon="content_copy" ButtonStyle="ButtonStyle.Secondary" Click="CopyToClipboard" class="rz-mr-2" />
|
|
<RadzenButton Text="Close" Click="Close" />
|
|
</FooterContent>
|
|
</RadzenDialog>
|
|
|
|
@code {
|
|
[Parameter] public bool Visible { get; set; }
|
|
[Parameter] public EventCallback<bool> VisibleChanged { get; set; }
|
|
[Parameter] public string? Title { get; set; }
|
|
[Parameter] public string? Sql { get; set; }
|
|
|
|
[Inject] private IJSRuntime JS { get; set; } = default!;
|
|
|
|
private string FormattedSql => FormatSql(Sql);
|
|
|
|
private static string FormatSql(string? sql)
|
|
{
|
|
if (string.IsNullOrWhiteSpace(sql))
|
|
return "";
|
|
|
|
// Basic SQL formatting - add line breaks before major clauses
|
|
return sql
|
|
.Replace(" SELECT ", "\nSELECT ")
|
|
.Replace(" FROM ", "\nFROM ")
|
|
.Replace(" WHERE ", "\nWHERE ")
|
|
.Replace(" AND ", "\n AND ")
|
|
.Replace(" OR ", "\n OR ")
|
|
.Replace(" LEFT ", "\nLEFT ")
|
|
.Replace(" RIGHT ", "\nRIGHT ")
|
|
.Replace(" INNER ", "\nINNER ")
|
|
.Replace(" OUTER ", "\nOUTER ")
|
|
.Replace(" JOIN ", " JOIN\n ")
|
|
.Replace(" ORDER BY ", "\nORDER BY ")
|
|
.Replace(" GROUP BY ", "\nGROUP BY ")
|
|
.Replace(" HAVING ", "\nHAVING ")
|
|
.Trim();
|
|
}
|
|
|
|
private async Task CopyToClipboard()
|
|
{
|
|
if (!string.IsNullOrWhiteSpace(Sql))
|
|
{
|
|
await JS.InvokeVoidAsync("navigator.clipboard.writeText", Sql);
|
|
}
|
|
}
|
|
|
|
private async Task Close()
|
|
{
|
|
await VisibleChanged.InvokeAsync(false);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 3: Build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Client
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/Components/Admin/
|
|
git commit -m "feat(client): add SqlQueryModal component"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 9: Create PipelineScheduleSection Component
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Client/Components/Admin/PipelineScheduleSection.razor`
|
|
|
|
**Step 1: Create PipelineScheduleSection.razor**
|
|
|
|
```razor
|
|
@namespace JdeScoping.Client.Components.Admin
|
|
@using JdeScoping.Core.ApiContracts.Pipelines
|
|
@using JdeScoping.Core.Models.Enums
|
|
|
|
<RadzenCard class="rz-mb-4">
|
|
<RadzenRow AlignItems="AlignItems.Center" class="rz-mb-3">
|
|
<RadzenColumn>
|
|
<RadzenText TextStyle="TextStyle.H5" class="rz-m-0">
|
|
@GetScheduleTypeName(ScheduleType) Refresh
|
|
</RadzenText>
|
|
</RadzenColumn>
|
|
<RadzenColumn Size="2" class="rz-text-align-end">
|
|
@if (Config?.Enabled == true)
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="Enabled" />
|
|
}
|
|
else
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Light" Text="Disabled" />
|
|
}
|
|
</RadzenColumn>
|
|
</RadzenRow>
|
|
|
|
@if (Config is not null)
|
|
{
|
|
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mb-2">Schedule Settings</RadzenText>
|
|
<table class="rz-mb-4" style="width: 100%; border-collapse: collapse;">
|
|
<thead>
|
|
<tr style="border-bottom: 1px solid #dee2e6;">
|
|
<th style="text-align: left; padding: 0.5rem;">Setting</th>
|
|
<th style="text-align: left; padding: 0.5rem;">Value</th>
|
|
<th style="text-align: left; padding: 0.5rem;">Source</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
<tr style="border-bottom: 1px solid #dee2e6;">
|
|
<td style="padding: 0.5rem;">Interval</td>
|
|
<td style="padding: 0.5rem;">@FormatInterval(Config.IntervalMinutes)</td>
|
|
<td style="padding: 0.5rem;">@(Config.IntervalIsOverride ? "Override" : "Default")</td>
|
|
</tr>
|
|
<tr style="border-bottom: 1px solid #dee2e6;">
|
|
<td style="padding: 0.5rem;">Pre-Purge</td>
|
|
<td style="padding: 0.5rem;">@(Config.PrePurge ? "Yes" : "No")</td>
|
|
<td style="padding: 0.5rem;">@(Config.PrePurgeIsOverride ? "Override" : "Default")</td>
|
|
</tr>
|
|
<tr style="border-bottom: 1px solid #dee2e6;">
|
|
<td style="padding: 0.5rem;">Re-Index</td>
|
|
<td style="padding: 0.5rem;">@(Config.ReIndex ? "Yes" : "No")</td>
|
|
<td style="padding: 0.5rem;">@(Config.ReIndexIsOverride ? "Override" : "Default")</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
@if (!string.IsNullOrWhiteSpace(QueryPreview))
|
|
{
|
|
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mb-2">Query</RadzenText>
|
|
<div style="background: #f5f5f5; padding: 0.75rem; border-radius: 4px; font-family: monospace; font-size: 0.875rem;">
|
|
@QueryPreview
|
|
</div>
|
|
@if (!string.IsNullOrWhiteSpace(FullQuery))
|
|
{
|
|
<RadzenButton Text="View Full Query" Icon="open_in_new" ButtonStyle="ButtonStyle.Light" Size="ButtonSize.Small"
|
|
Click="@(() => OnViewQuery.InvokeAsync(FullQuery))" class="rz-mt-2" />
|
|
}
|
|
}
|
|
}
|
|
</RadzenCard>
|
|
|
|
@code {
|
|
[Parameter] public UpdateTypes ScheduleType { get; set; }
|
|
[Parameter] public PipelineScheduleDto? Config { get; set; }
|
|
[Parameter] public string? QueryPreview { get; set; }
|
|
[Parameter] public string? FullQuery { get; set; }
|
|
[Parameter] public EventCallback<string> OnViewQuery { get; set; }
|
|
|
|
private static string GetScheduleTypeName(UpdateTypes type) => type switch
|
|
{
|
|
UpdateTypes.Mass => "Mass",
|
|
UpdateTypes.Daily => "Daily",
|
|
UpdateTypes.Hourly => "Hourly",
|
|
_ => type.ToString()
|
|
};
|
|
|
|
private static string FormatInterval(int minutes)
|
|
{
|
|
if (minutes >= 1440)
|
|
return $"{minutes / 1440} day(s) ({minutes} min)";
|
|
if (minutes >= 60)
|
|
return $"{minutes / 60} hour(s) ({minutes} min)";
|
|
return $"{minutes} minutes";
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Client
|
|
```
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/Components/Admin/PipelineScheduleSection.razor
|
|
git commit -m "feat(client): add PipelineScheduleSection component"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 10: Create PipelineViewer Page
|
|
|
|
**Files:**
|
|
- Create: `src/JdeScoping.Client/Pages/Admin/PipelineViewer.razor`
|
|
|
|
**Step 1: Create Admin directory**
|
|
|
|
```bash
|
|
mkdir -p src/JdeScoping.Client/Pages/Admin
|
|
```
|
|
|
|
**Step 2: Create PipelineViewer.razor**
|
|
|
|
```razor
|
|
@page "/admin/pipeline-viewer"
|
|
@attribute [Authorize]
|
|
@using JdeScoping.Client.Components.Admin
|
|
@using JdeScoping.Client.Components.Shared
|
|
@using JdeScoping.Core.ApiContracts.Pipelines
|
|
@using JdeScoping.Core.Models.Enums
|
|
@inject IPipelineApiClient PipelineApi
|
|
|
|
<PageTitle>Pipeline Configuration Viewer - JDE Scoping Tool</PageTitle>
|
|
|
|
<RadzenText TextStyle="TextStyle.H4" class="rz-mb-4">ETL Pipeline Configuration Viewer</RadzenText>
|
|
|
|
<!-- Pipeline Selector -->
|
|
<RadzenCard class="rz-mb-4">
|
|
<RadzenFormField Text="Select Pipeline" Style="width: 400px;">
|
|
<RadzenDropDown @bind-Value="_selectedPipeline" Data="@_pipelineNames" Style="width: 100%;"
|
|
Change="@OnPipelineChanged" Placeholder="Select a pipeline..." />
|
|
</RadzenFormField>
|
|
</RadzenCard>
|
|
|
|
@if (_isLoading)
|
|
{
|
|
<LoadingIndicator Message="Loading pipeline data..." />
|
|
}
|
|
else if (_config is not null)
|
|
{
|
|
<!-- Status Summary Table -->
|
|
<RadzenCard class="rz-mb-4">
|
|
<RadzenText TextStyle="TextStyle.H5" class="rz-mb-3">Schedule Status Summary</RadzenText>
|
|
<RadzenDataGrid Data="@_statuses" TItem="PipelineScheduleStatusDto">
|
|
<Columns>
|
|
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Property="ScheduleType" Title="Type" Width="100px">
|
|
<Template Context="item">@item.ScheduleType.ToString()</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Last Run" Width="160px">
|
|
<Template Context="item">@(item.LastRun?.ToString("MM/dd/yyyy hh:mm tt") ?? "Never")</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Success?" Width="80px" TextAlign="TextAlign.Center">
|
|
<Template Context="item">
|
|
@if (item.LastRun.HasValue)
|
|
{
|
|
@if (item.LastRunWasSuccessful)
|
|
{
|
|
<RadzenIcon Icon="check_circle" Style="color: green;" />
|
|
}
|
|
else
|
|
{
|
|
<RadzenIcon Icon="cancel" Style="color: red;" />
|
|
}
|
|
}
|
|
</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Next Required" Width="160px">
|
|
<Template Context="item">@(item.NextRequiredRun?.ToString("MM/dd/yyyy hh:mm tt") ?? "N/A")</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Status" Width="100px" TextAlign="TextAlign.Center">
|
|
<Template Context="item">
|
|
@if (item.IsOverdue)
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Warning" Text="Overdue" />
|
|
}
|
|
else
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="OK" />
|
|
}
|
|
</Template>
|
|
</RadzenDataGridColumn>
|
|
</Columns>
|
|
</RadzenDataGrid>
|
|
</RadzenCard>
|
|
|
|
<!-- Execution History Table -->
|
|
<RadzenCard class="rz-mb-4">
|
|
<RadzenText TextStyle="TextStyle.H5" class="rz-mb-3">Recent Execution History</RadzenText>
|
|
<RadzenDataGrid Data="@_executions" TItem="PipelineExecutionDto" AllowSorting="true" PageSize="10" AllowPaging="true">
|
|
<Columns>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Property="ScheduleType" Title="Type" Width="100px">
|
|
<Template Context="item">@item.ScheduleType.ToString()</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Start" Width="160px">
|
|
<Template Context="item">@item.StartTime.ToString("MM/dd/yyyy hh:mm tt")</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="End" Width="160px">
|
|
<Template Context="item">@(item.EndTime?.ToString("MM/dd/yyyy hh:mm tt") ?? "In Progress")</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Duration" Width="100px">
|
|
<Template Context="item">@FormatDuration(item.Duration)</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Property="RecordCount" Title="Records" Width="100px" TextAlign="TextAlign.Right">
|
|
<Template Context="item">@item.RecordCount.ToString("N0")</Template>
|
|
</RadzenDataGridColumn>
|
|
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Result" Width="100px" TextAlign="TextAlign.Center">
|
|
<Template Context="item">
|
|
@if (item.WasSuccessful)
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="Success" />
|
|
}
|
|
else
|
|
{
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Danger" Text="Failed" />
|
|
}
|
|
</Template>
|
|
</RadzenDataGridColumn>
|
|
</Columns>
|
|
</RadzenDataGrid>
|
|
</RadzenCard>
|
|
|
|
<!-- Common Pipeline Info -->
|
|
<RadzenRow Gap="1rem" class="rz-mb-4">
|
|
<!-- Source Card -->
|
|
<RadzenColumn Size="4">
|
|
<RadzenCard Style="height: 100%;">
|
|
<RadzenText TextStyle="TextStyle.H6" class="rz-mb-2">Source</RadzenText>
|
|
<p><strong>Connection:</strong>
|
|
@switch (_config.Source.Connection.ToLower())
|
|
{
|
|
case "jde":
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Info" Text="JDE" />
|
|
break;
|
|
case "cms":
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="CMS" />
|
|
break;
|
|
case "giw":
|
|
<RadzenBadge BadgeStyle="BadgeStyle.Warning" Text="GIW" />
|
|
break;
|
|
default:
|
|
<RadzenBadge Text="@_config.Source.Connection" />
|
|
break;
|
|
}
|
|
</p>
|
|
@if (_config.Source.Parameters.Count > 0)
|
|
{
|
|
<p><strong>Parameters:</strong></p>
|
|
<ul>
|
|
@foreach (var param in _config.Source.Parameters)
|
|
{
|
|
<li>@param.Name (@(param.Format ?? "default"))</li>
|
|
}
|
|
</ul>
|
|
}
|
|
</RadzenCard>
|
|
</RadzenColumn>
|
|
|
|
<!-- Destination Card -->
|
|
<RadzenColumn Size="4">
|
|
<RadzenCard Style="height: 100%;">
|
|
<RadzenText TextStyle="TextStyle.H6" class="rz-mb-2">Destination</RadzenText>
|
|
<p><strong>Table:</strong> @_config.Destination.Table</p>
|
|
<p><strong>Operation:</strong>
|
|
<RadzenBadge BadgeStyle="@(_config.Destination.OperationType == "BulkMerge" ? BadgeStyle.Primary : BadgeStyle.Secondary)"
|
|
Text="@_config.Destination.OperationType" />
|
|
</p>
|
|
@if (_config.Destination.MatchColumns?.Count > 0)
|
|
{
|
|
<p><strong>Match Columns:</strong> @string.Join(", ", _config.Destination.MatchColumns)</p>
|
|
}
|
|
@if (_config.Destination.ExcludeFromUpdate?.Count > 0)
|
|
{
|
|
<p><strong>Exclude:</strong> @string.Join(", ", _config.Destination.ExcludeFromUpdate)</p>
|
|
}
|
|
</RadzenCard>
|
|
</RadzenColumn>
|
|
|
|
<!-- Scripts Card -->
|
|
<RadzenColumn Size="4">
|
|
<RadzenCard Style="height: 100%;">
|
|
<RadzenText TextStyle="TextStyle.H6" class="rz-mb-2">Scripts</RadzenText>
|
|
<p><strong>Pre-Scripts:</strong> @_config.PreScriptCount</p>
|
|
<p><strong>Post-Scripts:</strong> @_config.PostScriptCount</p>
|
|
@if (_config.PreScripts?.Count > 0)
|
|
{
|
|
<RadzenText TextStyle="TextStyle.Subtitle2" class="rz-mt-2">Pre-Scripts:</RadzenText>
|
|
@for (int i = 0; i < _config.PreScripts.Count; i++)
|
|
{
|
|
var script = _config.PreScripts[i];
|
|
var index = i + 1;
|
|
<div>
|
|
<RadzenButton Text="@($"Script {index}")" Icon="code" ButtonStyle="ButtonStyle.Light" Size="ButtonSize.Small"
|
|
Click="@(() => ShowSqlModal($"Pre-Script {index}", script))" />
|
|
</div>
|
|
}
|
|
}
|
|
@if (_config.PostScripts?.Count > 0)
|
|
{
|
|
<RadzenText TextStyle="TextStyle.Subtitle2" class="rz-mt-2">Post-Scripts:</RadzenText>
|
|
@for (int i = 0; i < _config.PostScripts.Count; i++)
|
|
{
|
|
var script = _config.PostScripts[i];
|
|
var index = i + 1;
|
|
<div>
|
|
<RadzenButton Text="@($"Script {index}")" Icon="code" ButtonStyle="ButtonStyle.Light" Size="ButtonSize.Small"
|
|
Click="@(() => ShowSqlModal($"Post-Script {index}", script))" />
|
|
</div>
|
|
}
|
|
}
|
|
</RadzenCard>
|
|
</RadzenColumn>
|
|
</RadzenRow>
|
|
|
|
<!-- Schedule Sections -->
|
|
<PipelineScheduleSection ScheduleType="UpdateTypes.Mass" Config="@_config.Schedules.Mass"
|
|
QueryPreview="@_config.Source.MassQueryPreview" FullQuery="@_config.Source.MassQuery"
|
|
OnViewQuery="@(q => ShowSqlModal("Mass Query", q))" />
|
|
|
|
<PipelineScheduleSection ScheduleType="UpdateTypes.Daily" Config="@_config.Schedules.Daily"
|
|
QueryPreview="@_config.Source.QueryPreview" FullQuery="@_config.Source.Query"
|
|
OnViewQuery="@(q => ShowSqlModal("Daily Query", q))" />
|
|
|
|
<PipelineScheduleSection ScheduleType="UpdateTypes.Hourly" Config="@_config.Schedules.Hourly"
|
|
QueryPreview="@_config.Source.QueryPreview" FullQuery="@_config.Source.Query"
|
|
OnViewQuery="@(q => ShowSqlModal("Hourly Query", q))" />
|
|
}
|
|
|
|
<SqlQueryModal @bind-Visible="_showSqlModal" Title="@_sqlModalTitle" Sql="@_sqlModalContent" />
|
|
|
|
@code {
|
|
private List<string> _pipelineNames = [];
|
|
private string? _selectedPipeline;
|
|
private bool _isLoading;
|
|
private PipelineConfigDto? _config;
|
|
private List<PipelineScheduleStatusDto> _statuses = [];
|
|
private List<PipelineExecutionDto> _executions = [];
|
|
|
|
private bool _showSqlModal;
|
|
private string? _sqlModalTitle;
|
|
private string? _sqlModalContent;
|
|
|
|
protected override async Task OnInitializedAsync()
|
|
{
|
|
var result = await PipelineApi.GetPipelineNamesAsync();
|
|
if (result.TryGetValue(out var response))
|
|
{
|
|
_pipelineNames = response.PipelineNames;
|
|
}
|
|
}
|
|
|
|
private async Task OnPipelineChanged()
|
|
{
|
|
if (string.IsNullOrWhiteSpace(_selectedPipeline))
|
|
{
|
|
_config = null;
|
|
_statuses = [];
|
|
_executions = [];
|
|
return;
|
|
}
|
|
|
|
_isLoading = true;
|
|
try
|
|
{
|
|
var configTask = PipelineApi.GetPipelineAsync(_selectedPipeline);
|
|
var statusTask = PipelineApi.GetStatusAsync(_selectedPipeline);
|
|
var executionsTask = PipelineApi.GetExecutionsAsync(_selectedPipeline);
|
|
|
|
await Task.WhenAll(configTask, statusTask, executionsTask);
|
|
|
|
if (configTask.Result.TryGetValue(out var config))
|
|
_config = config;
|
|
|
|
if (statusTask.Result.TryGetValue(out var status))
|
|
_statuses = status.Statuses;
|
|
|
|
if (executionsTask.Result.TryGetValue(out var execs))
|
|
_executions = execs.Executions;
|
|
}
|
|
finally
|
|
{
|
|
_isLoading = false;
|
|
}
|
|
}
|
|
|
|
private void ShowSqlModal(string title, string sql)
|
|
{
|
|
_sqlModalTitle = $"{title} - {_selectedPipeline}";
|
|
_sqlModalContent = sql;
|
|
_showSqlModal = true;
|
|
}
|
|
|
|
private static string FormatDuration(TimeSpan? duration)
|
|
{
|
|
if (!duration.HasValue) return "-";
|
|
var d = duration.Value;
|
|
if (d.TotalHours >= 1) return $"{d.Hours}h {d.Minutes}m";
|
|
if (d.TotalMinutes >= 1) return $"{d.Minutes}m {d.Seconds}s";
|
|
return $"{d.Seconds}s";
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 3: Build**
|
|
|
|
```bash
|
|
dotnet build src/JdeScoping.Client
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/Pages/Admin/
|
|
git commit -m "feat(client): add PipelineViewer page"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 11: Register Services and Build Full Solution
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.Client/Program.cs`
|
|
|
|
**Step 1: Register PipelineApiClient**
|
|
|
|
Add to Program.cs DI section:
|
|
|
|
```csharp
|
|
builder.Services.AddScoped<IPipelineApiClient, PipelineApiClient>();
|
|
```
|
|
|
|
**Step 2: Build full solution**
|
|
|
|
```bash
|
|
dotnet build
|
|
```
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
dotnet test
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/Program.cs
|
|
git commit -m "feat(client): register PipelineApiClient in DI"
|
|
```
|
|
|
|
---
|
|
|
|
## Task 12: Add Navigation Link (Optional)
|
|
|
|
**Files:**
|
|
- Modify: `src/JdeScoping.Client/Layout/MainLayout.razor` (or NavMenu)
|
|
|
|
**Step 1: Add link to admin section**
|
|
|
|
Find navigation and add:
|
|
|
|
```razor
|
|
<RadzenPanelMenuItem Text="Pipeline Viewer" Path="admin/pipeline-viewer" Icon="settings_ethernet" />
|
|
```
|
|
|
|
**Step 2: Build and test manually**
|
|
|
|
```bash
|
|
dotnet run --project src/JdeScoping.Host
|
|
```
|
|
|
|
Navigate to `/admin/pipeline-viewer` in browser.
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add src/JdeScoping.Client/
|
|
git commit -m "feat(client): add pipeline viewer to navigation"
|
|
```
|
|
|
|
---
|
|
|
|
## Final Verification
|
|
|
|
```bash
|
|
# Build
|
|
dotnet build
|
|
|
|
# Run tests
|
|
dotnet test
|
|
|
|
# Start application
|
|
dotnet run --project src/JdeScoping.Host
|
|
```
|
|
|
|
Navigate to `/admin/pipeline-viewer`, select a pipeline, and verify:
|
|
- Pipeline list loads
|
|
- Status summary shows
|
|
- Execution history shows
|
|
- Configuration cards display
|
|
- Schedule sections show with Default/Override indicators
|
|
- SQL modal works with copy button
|