refactor(webui): remove pipeline viewer feature
Remove the read-only pipeline viewer from the web UI: - Delete PipelineViewer.razor page and supporting components - Delete PipelineController and PipelineMapper from API - Delete Pipeline DTOs from Core - Delete PipelineApiClient from Client - Remove navigation link and DI registrations - Delete obsolete plan documents The ConfigManager utility retains pipeline editing capabilities.
This commit is contained in:
@@ -1,488 +0,0 @@
|
||||
# ETL Pipeline Viewer Component Design
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Build a Blazor component that visualizes ETL pipeline configuration and execution status.
|
||||
|
||||
**Architecture:** Server-side API loads pipeline config and exposes DTOs; Blazor client consumes via typed HttpClient following existing ApiClientBase pattern.
|
||||
|
||||
**Tech Stack:** Blazor WebAssembly, Radzen components, existing DataSync config types, existing DataUpdateRepository
|
||||
|
||||
---
|
||||
|
||||
## Codex Review Feedback (Addressed)
|
||||
|
||||
| Issue | Resolution |
|
||||
|-------|------------|
|
||||
| Client loading pipelines.json directly breaks CLEAN | API loads config, returns DTO to client |
|
||||
| Duplicate config records diverge from DataSync | Reuse existing DataSync Configuration types |
|
||||
| GetRecentUpdatesAsync missing UpdateType filter | Add UpdateType parameter to signature |
|
||||
| Status/overdue logic diverges from DataSync | Reuse existing `DataUpdateRepository.IsOverdue()` |
|
||||
| Only LastSuccessfulRun, hiding failures | Add LastRun + LastRunWasSuccessful |
|
||||
| API patterns don't match ApiRoutes + ApiClientBase | Add `ApiRoutes.Pipelines`, implement `PipelineApiClient` |
|
||||
| Duration non-nullable for in-progress | Make Duration nullable |
|
||||
| ScheduleType as string | Use `UpdateTypes` enum |
|
||||
| Missing authorization | Add `[Authorize]` to page and controller |
|
||||
|
||||
---
|
||||
|
||||
## Component Structure
|
||||
|
||||
### Page Layout
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ ETL Pipeline Configuration Viewer [Authorize] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Pipeline: [Dropdown - Alphabetical list] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ SCHEDULE STATUS SUMMARY │
|
||||
│ ┌─────────┬────────────┬────────────┬─────────────┬───────┐ │
|
||||
│ │ Type │ Last Run │ Success? │ Next Req. │Status │ │
|
||||
│ ├─────────┼────────────┼────────────┼─────────────┼───────┤ │
|
||||
│ │ Mass │ 01/05 02:00│ ✓ │ 01/12 02:00 │ OK │ │
|
||||
│ │ Daily │ 01/07 01:00│ ✓ │ 01/08 01:00 │ OK │ │
|
||||
│ │ Hourly │ 01/07 10:00│ ✗ │ 01/07 11:00 │ Over │ │
|
||||
│ └─────────┴────────────┴────────────┴─────────────┴───────┘ │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ RECENT EXECUTION HISTORY (Last 10 per type) │
|
||||
│ ┌─────────┬────────────┬──────────┬─────────┬─────┬───────┐ │
|
||||
│ │ Type │ Start │ End │ Duration│ Rows│ Result│ │
|
||||
│ └─────────┴────────────┴──────────┴─────────┴─────┴───────┘ │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ COMMON PIPELINE INFO │
|
||||
│ [Source Card] [Destination Card] [Scripts Card] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ MASS REFRESH [Enabled] │
|
||||
│ [Schedule Settings] [Query Preview + Modal] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ DAILY REFRESH [Enabled] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ HOURLY REFRESH [Enabled] │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### CLEAN Architecture Layers
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PRESENTATION (JdeScoping.Client) │
|
||||
│ - PipelineViewer.razor (page) │
|
||||
│ - PipelineScheduleSection.razor (component) │
|
||||
│ - SqlQueryModal.razor (component) │
|
||||
│ - PipelineApiClient : ApiClientBase │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ HTTP via ApiRoutes.Pipelines
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ APPLICATION (JdeScoping.Api) │
|
||||
│ - PipelineController │
|
||||
│ - Returns DTOs from Core.ApiContracts │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ Depends on
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ INFRASTRUCTURE (JdeScoping.DataSync) │
|
||||
│ - IEtlPipelineFactory (existing - loads pipelines.json) │
|
||||
│ - IDataUpdateRepository (add GetRecentUpdatesAsync) │
|
||||
│ - Reuse existing Configuration/* types │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ Depends on
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ DOMAIN (JdeScoping.Core) │
|
||||
│ - ApiRoutes.Pipelines (route constants) │
|
||||
│ - ApiContracts.Pipelines/* (DTOs) │
|
||||
│ - Models.Enums.UpdateTypes (existing) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
### Core Project (`src/JdeScoping.Core/`)
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `ApiContracts/ApiRoutes.cs` | Modify | Add `Pipelines` routes |
|
||||
| `ApiContracts/Pipelines/PipelineListResponse.cs` | Create | List of pipeline names |
|
||||
| `ApiContracts/Pipelines/PipelineConfigDto.cs` | Create | Pipeline config DTO (sanitized) |
|
||||
| `ApiContracts/Pipelines/PipelineStatusDto.cs` | Create | Schedule status DTO |
|
||||
| `ApiContracts/Pipelines/PipelineExecutionDto.cs` | Create | Execution history DTO |
|
||||
|
||||
### DataSync Project (`src/JdeScoping.DataSync/`)
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `Contracts/IDataUpdateRepository.cs` | Modify | Add `GetRecentUpdatesAsync` |
|
||||
| `Services/DataUpdateRepository.cs` | Modify | Implement new method |
|
||||
|
||||
### API Project (`src/JdeScoping.Api/`)
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `Controllers/PipelineController.cs` | Create | API endpoints |
|
||||
|
||||
### Client Project (`src/JdeScoping.Client/`)
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `Services/IPipelineApiClient.cs` | Create | Client interface |
|
||||
| `Services/PipelineApiClient.cs` | Create | Client implementation |
|
||||
| `Pages/Admin/PipelineViewer.razor` | Create | Main page |
|
||||
| `Components/Admin/PipelineScheduleSection.razor` | Create | Schedule section |
|
||||
| `Components/Admin/SqlQueryModal.razor` | Create | SQL modal |
|
||||
|
||||
---
|
||||
|
||||
## API Routes (add to `ApiRoutes.cs`)
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Routes for pipeline configuration API endpoints.
|
||||
/// </summary>
|
||||
public static class Pipelines
|
||||
{
|
||||
/// <summary>Base route for pipeline endpoints.</summary>
|
||||
public const string Base = "api/pipelines";
|
||||
|
||||
/// <summary>Route template for getting a pipeline by name.</summary>
|
||||
public const string ByName = "{name}";
|
||||
|
||||
/// <summary>Route template for getting pipeline status.</summary>
|
||||
public const string Status = "{name}/status";
|
||||
|
||||
/// <summary>Route template for getting pipeline executions.</summary>
|
||||
public const string Executions = "{name}/executions";
|
||||
|
||||
/// <summary>Builds the route to get a specific pipeline config.</summary>
|
||||
public static string GetByName(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}";
|
||||
|
||||
/// <summary>Builds the route to get pipeline status.</summary>
|
||||
public static string GetStatus(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}/status";
|
||||
|
||||
/// <summary>Builds the route to get pipeline executions.</summary>
|
||||
public static string GetExecutions(string name, int count = 10) =>
|
||||
$"api/pipelines/{Uri.EscapeDataString(name)}/executions?count={count}";
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## DTOs (in `Core/ApiContracts/Pipelines/`)
|
||||
|
||||
### PipelineListResponse.cs
|
||||
|
||||
```csharp
|
||||
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
||||
|
||||
/// <summary>
|
||||
/// Response containing list of available pipeline names.
|
||||
/// </summary>
|
||||
public record PipelineListResponse(List<string> PipelineNames);
|
||||
```
|
||||
|
||||
### PipelineConfigDto.cs
|
||||
|
||||
```csharp
|
||||
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline configuration DTO for display (sanitized - no raw SQL exposed by default).
|
||||
/// </summary>
|
||||
public record PipelineConfigDto(
|
||||
string Name,
|
||||
PipelineSourceDto Source,
|
||||
PipelineDestinationDto Destination,
|
||||
PipelineSchedulesDto Schedules,
|
||||
int PreScriptCount,
|
||||
int PostScriptCount,
|
||||
List<string>? PreScripts, // Only populated if user has admin role
|
||||
List<string>? PostScripts); // Only populated if user has admin role
|
||||
|
||||
public record PipelineSourceDto(
|
||||
string Connection,
|
||||
string? QueryPreview, // First 100 chars
|
||||
string? MassQueryPreview, // First 100 chars
|
||||
string? Query, // Full query - only if admin
|
||||
string? MassQuery, // Full query - only if admin
|
||||
List<PipelineParameterDto> Parameters);
|
||||
|
||||
public record PipelineParameterDto(
|
||||
string Name,
|
||||
string? Format,
|
||||
string Source);
|
||||
|
||||
public record PipelineDestinationDto(
|
||||
string Table,
|
||||
string OperationType, // "BulkMerge" or "BulkImport"
|
||||
List<string>? MatchColumns,
|
||||
List<string>? ExcludeFromUpdate);
|
||||
|
||||
public record PipelineSchedulesDto(
|
||||
PipelineScheduleDto Mass,
|
||||
PipelineScheduleDto Daily,
|
||||
PipelineScheduleDto Hourly);
|
||||
|
||||
public record PipelineScheduleDto(
|
||||
bool Enabled,
|
||||
int IntervalMinutes,
|
||||
bool PrePurge,
|
||||
bool ReIndex,
|
||||
bool IntervalIsOverride,
|
||||
bool PrePurgeIsOverride,
|
||||
bool ReIndexIsOverride);
|
||||
```
|
||||
|
||||
### PipelineStatusDto.cs
|
||||
|
||||
```csharp
|
||||
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
||||
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline schedule status for each update type.
|
||||
/// </summary>
|
||||
public record PipelineStatusResponse(List<PipelineScheduleStatusDto> Statuses);
|
||||
|
||||
public record PipelineScheduleStatusDto(
|
||||
UpdateTypes ScheduleType,
|
||||
DateTime? LastRun,
|
||||
bool LastRunWasSuccessful,
|
||||
DateTime? LastSuccessfulRun,
|
||||
DateTime? NextRequiredRun,
|
||||
bool IsOverdue,
|
||||
int IntervalMinutes);
|
||||
```
|
||||
|
||||
### PipelineExecutionDto.cs
|
||||
|
||||
```csharp
|
||||
namespace JdeScoping.Core.ApiContracts.Pipelines;
|
||||
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline execution history.
|
||||
/// </summary>
|
||||
public record PipelineExecutionsResponse(List<PipelineExecutionDto> Executions);
|
||||
|
||||
public record PipelineExecutionDto(
|
||||
UpdateTypes ScheduleType,
|
||||
DateTime StartTime,
|
||||
DateTime? EndTime,
|
||||
TimeSpan? Duration, // Nullable for in-progress runs
|
||||
long RecordCount,
|
||||
bool WasSuccessful);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Repository Update (IDataUpdateRepository)
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Gets the last N execution records for a specific table and optional update type.
|
||||
/// </summary>
|
||||
/// <param name="tableName">The table name to filter by.</param>
|
||||
/// <param name="updateType">Optional update type filter. If null, returns all types.</param>
|
||||
/// <param name="count">Maximum records to return per update type.</param>
|
||||
/// <param name="cancellationToken">Cancellation token.</param>
|
||||
/// <returns>List of DataUpdate records ordered by StartDt descending.</returns>
|
||||
Task<List<DataUpdate>> GetRecentUpdatesAsync(
|
||||
string tableName,
|
||||
UpdateTypes? updateType = null,
|
||||
int count = 10,
|
||||
CancellationToken cancellationToken = default);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the last run (successful or not) for each update type for a table.
|
||||
/// </summary>
|
||||
/// <param name="tableName">The table name.</param>
|
||||
/// <param name="cancellationToken">Cancellation token.</param>
|
||||
/// <returns>Dictionary keyed by UpdateType.</returns>
|
||||
Task<Dictionary<UpdateTypes, DataUpdate>> GetLastRunsAsync(
|
||||
string tableName,
|
||||
CancellationToken cancellationToken = default);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Controller
|
||||
|
||||
```csharp
|
||||
[ApiController]
|
||||
[Route(ApiRoutes.Pipelines.Base)]
|
||||
[Authorize]
|
||||
public class PipelineController : ControllerBase
|
||||
{
|
||||
private readonly IEtlPipelineFactory _pipelineFactory;
|
||||
private readonly IDataUpdateRepository _dataUpdateRepository;
|
||||
|
||||
[HttpGet]
|
||||
public ActionResult<PipelineListResponse> GetPipelineNames()
|
||||
{
|
||||
var names = _pipelineFactory.GetAvailableTables()
|
||||
.OrderBy(n => n)
|
||||
.ToList();
|
||||
return Ok(new PipelineListResponse(names));
|
||||
}
|
||||
|
||||
[HttpGet(ApiRoutes.Pipelines.ByName)]
|
||||
public ActionResult<PipelineConfigDto> GetPipeline(string name)
|
||||
{
|
||||
// Map from internal config to DTO
|
||||
// Truncate queries for preview, include full if admin
|
||||
}
|
||||
|
||||
[HttpGet(ApiRoutes.Pipelines.Status)]
|
||||
public async Task<ActionResult<PipelineStatusResponse>> GetStatus(string name)
|
||||
{
|
||||
// Get last runs, calculate IsOverdue using DataUpdateRepository.IsOverdue()
|
||||
}
|
||||
|
||||
[HttpGet(ApiRoutes.Pipelines.Executions)]
|
||||
public async Task<ActionResult<PipelineExecutionsResponse>> GetExecutions(
|
||||
string name, [FromQuery] int count = 10)
|
||||
{
|
||||
// Get recent updates from repository
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Client Implementation
|
||||
|
||||
### PipelineApiClient.cs
|
||||
|
||||
```csharp
|
||||
public interface IPipelineApiClient
|
||||
{
|
||||
Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default);
|
||||
Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default);
|
||||
Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default);
|
||||
Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 10, CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public class PipelineApiClient : ApiClientBase, IPipelineApiClient
|
||||
{
|
||||
public PipelineApiClient(HttpClient httpClient) : base(httpClient) { }
|
||||
|
||||
public Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default)
|
||||
=> GetAsync<PipelineListResponse>(ApiRoutes.Pipelines.Base, ct);
|
||||
|
||||
public Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineConfigDto>(ApiRoutes.Pipelines.GetByName(name), ct);
|
||||
|
||||
public Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineStatusResponse>(ApiRoutes.Pipelines.GetStatus(name), ct);
|
||||
|
||||
public Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 10, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineExecutionsResponse>(ApiRoutes.Pipelines.GetExecutions(name, count), ct);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Blazor Components
|
||||
|
||||
### PipelineViewer.razor Structure
|
||||
|
||||
```razor
|
||||
@page "/admin/pipeline-viewer"
|
||||
@attribute [Authorize]
|
||||
@inject IPipelineApiClient PipelineApi
|
||||
|
||||
<PageTitle>Pipeline Configuration Viewer</PageTitle>
|
||||
|
||||
<RadzenText TextStyle="TextStyle.H4">ETL Pipeline Configuration Viewer</RadzenText>
|
||||
|
||||
<!-- Pipeline Selector -->
|
||||
<RadzenDropDown @bind-Value="_selectedPipeline"
|
||||
Data="_pipelineNames"
|
||||
Change="OnPipelineChanged" />
|
||||
|
||||
@if (_config is not null)
|
||||
{
|
||||
<!-- Status Summary Table -->
|
||||
<!-- Execution History Table -->
|
||||
<!-- Common Info Cards -->
|
||||
<!-- Schedule Sections (Mass, Daily, Hourly) -->
|
||||
}
|
||||
|
||||
<SqlQueryModal @bind-Visible="_showSqlModal" Title="@_sqlModalTitle" Sql="@_sqlModalContent" />
|
||||
```
|
||||
|
||||
### SqlQueryModal.razor
|
||||
|
||||
- Large modal (80% width)
|
||||
- `<pre><code>` with monospace font
|
||||
- Copy to clipboard button
|
||||
- Basic SQL formatting (line breaks at clauses)
|
||||
|
||||
### PipelineScheduleSection.razor
|
||||
|
||||
- Props: ScheduleType (UpdateTypes), Config (PipelineScheduleDto), Query, OnViewQuery callback
|
||||
- Shows enabled badge
|
||||
- Settings table with Default/Override indicators
|
||||
- Query preview with "View Full" button
|
||||
|
||||
---
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Override Detection (in API)
|
||||
|
||||
```csharp
|
||||
bool IsOverride<T>(T? pipelineValue, T defaultValue) where T : struct =>
|
||||
pipelineValue.HasValue && !pipelineValue.Value.Equals(defaultValue);
|
||||
```
|
||||
|
||||
### Overdue Calculation (reuse existing)
|
||||
|
||||
```csharp
|
||||
// Use DataUpdateRepository.IsOverdue() which includes 50% grace period
|
||||
var isOverdue = DataUpdateRepository.IsOverdue(
|
||||
lastSuccessfulRun, tableName, updateType, customIntervals);
|
||||
```
|
||||
|
||||
### Connection Type Badge Colors
|
||||
|
||||
| Connection | Badge Style |
|
||||
|------------|-------------|
|
||||
| jde | `BadgeStyle.Info` (blue) |
|
||||
| cms | `BadgeStyle.Success` (green) |
|
||||
| giw | `BadgeStyle.Warning` (orange) |
|
||||
|
||||
### SQL Query Truncation
|
||||
|
||||
```csharp
|
||||
static string Truncate(string? sql, int maxLength = 100) =>
|
||||
sql is null ? "" :
|
||||
sql.Length <= maxLength ? sql : sql[..maxLength] + "...";
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authorization**: Page and API require `[Authorize]`
|
||||
2. **SQL Visibility**: Full SQL/scripts only exposed if user has appropriate permissions (consider admin-only)
|
||||
3. **No write operations**: This is read-only, no mutations exposed
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Radzen.Blazor (existing)
|
||||
- Existing `IEtlPipelineFactory` and `IDataUpdateRepository`
|
||||
- Existing `pipelines.json` configuration
|
||||
- Existing `UpdateTypes` enum
|
||||
- Existing `ApiClientBase` pattern
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,329 @@
|
||||
# Remove Pipeline Viewer Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Remove the pipeline viewer feature from the web UI and clean up all supporting code (controllers, DTOs, client services, routes).
|
||||
|
||||
**Architecture:** Delete files in dependency order (DTOs/interfaces first would break compilation), so we delete leaf components first (views, client), then API layer, then shared contracts/DTOs. Modify DI registrations and navigation.
|
||||
|
||||
**Tech Stack:** Blazor WebAssembly, ASP.NET Core API, C#
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Remove Client Components (Blazor Pages)
|
||||
|
||||
**Files:**
|
||||
- Delete: `src/JdeScoping.Client/Pages/Admin/PipelineViewer.razor`
|
||||
- Delete: `src/JdeScoping.Client/Components/Admin/PipelineScheduleSection.razor`
|
||||
- Delete: `src/JdeScoping.Client/Components/Admin/SqlQueryModal.razor`
|
||||
|
||||
**Step 1: Delete PipelineViewer.razor**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Client/Pages/Admin/PipelineViewer.razor
|
||||
```
|
||||
|
||||
**Step 2: Delete PipelineScheduleSection.razor**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Client/Components/Admin/PipelineScheduleSection.razor
|
||||
```
|
||||
|
||||
**Step 3: Delete SqlQueryModal.razor (if exists)**
|
||||
|
||||
```bash
|
||||
rm -f src/JdeScoping.Client/Components/Admin/SqlQueryModal.razor
|
||||
```
|
||||
|
||||
**Step 4: Verify files deleted**
|
||||
|
||||
```bash
|
||||
ls src/JdeScoping.Client/Pages/Admin/ | grep -i pipeline || echo "No pipeline pages remain"
|
||||
ls src/JdeScoping.Client/Components/Admin/ | grep -i pipeline || echo "No pipeline components remain"
|
||||
```
|
||||
|
||||
Expected: No pipeline-related files in Admin folders.
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Remove Navigation Link
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/JdeScoping.Client/Layout/MainLayout.razor:16`
|
||||
|
||||
**Step 1: Remove the pipeline viewer nav link**
|
||||
|
||||
In `MainLayout.razor`, delete this line (around line 16):
|
||||
```razor
|
||||
<NavLink class="nav-link" href="/admin/pipeline-viewer">Pipeline Viewer</NavLink>
|
||||
```
|
||||
|
||||
**Step 2: Verify the change**
|
||||
|
||||
```bash
|
||||
grep -n "pipeline-viewer" src/JdeScoping.Client/Layout/MainLayout.razor || echo "Nav link removed"
|
||||
```
|
||||
|
||||
Expected: "Nav link removed"
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Remove Client Service
|
||||
|
||||
**Files:**
|
||||
- Delete: `src/JdeScoping.Client/Services/PipelineApiClient.cs`
|
||||
- Modify: `src/JdeScoping.Client/Program.cs:58`
|
||||
|
||||
**Step 1: Delete PipelineApiClient.cs**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Client/Services/PipelineApiClient.cs
|
||||
```
|
||||
|
||||
**Step 2: Remove DI registration from Program.cs**
|
||||
|
||||
In `src/JdeScoping.Client/Program.cs`, delete this line (around line 58):
|
||||
```csharp
|
||||
builder.Services.AddScoped<IPipelineApiClient, PipelineApiClient>();
|
||||
```
|
||||
|
||||
**Step 3: Verify the changes**
|
||||
|
||||
```bash
|
||||
grep -n "PipelineApiClient" src/JdeScoping.Client/Program.cs || echo "DI registration removed"
|
||||
ls src/JdeScoping.Client/Services/PipelineApiClient.cs 2>/dev/null || echo "File deleted"
|
||||
```
|
||||
|
||||
Expected: Both checks show removal confirmed.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Remove API Controller and Mapper
|
||||
|
||||
**Files:**
|
||||
- Delete: `src/JdeScoping.Api/Controllers/PipelineController.cs`
|
||||
- Delete: `src/JdeScoping.Api/Mapping/PipelineMapper.cs`
|
||||
- Delete: `src/JdeScoping.Api/Mapping/IPipelineMapper.cs`
|
||||
- Modify: `src/JdeScoping.Api/DependencyInjection.cs:43`
|
||||
|
||||
**Step 1: Delete PipelineController.cs**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Api/Controllers/PipelineController.cs
|
||||
```
|
||||
|
||||
**Step 2: Delete PipelineMapper.cs**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Api/Mapping/PipelineMapper.cs
|
||||
```
|
||||
|
||||
**Step 3: Delete IPipelineMapper.cs**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Api/Mapping/IPipelineMapper.cs
|
||||
```
|
||||
|
||||
**Step 4: Remove DI registration from DependencyInjection.cs**
|
||||
|
||||
In `src/JdeScoping.Api/DependencyInjection.cs`, delete this line (around line 43):
|
||||
```csharp
|
||||
services.AddSingleton<IPipelineMapper, PipelineMapper>();
|
||||
```
|
||||
|
||||
**Step 5: Verify the changes**
|
||||
|
||||
```bash
|
||||
ls src/JdeScoping.Api/Controllers/PipelineController.cs 2>/dev/null || echo "Controller deleted"
|
||||
ls src/JdeScoping.Api/Mapping/*Pipeline* 2>/dev/null || echo "Mapper files deleted"
|
||||
grep -n "PipelineMapper" src/JdeScoping.Api/DependencyInjection.cs || echo "DI registration removed"
|
||||
```
|
||||
|
||||
Expected: All three checks confirm removal.
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Remove Core DTOs and Interface
|
||||
|
||||
**Files:**
|
||||
- Delete: `src/JdeScoping.Core/Models/Pipelines/` (entire folder)
|
||||
- Delete: `src/JdeScoping.Core/ApiContracts/IPipelineApiClient.cs`
|
||||
|
||||
**Step 1: Delete the Pipelines DTO folder**
|
||||
|
||||
```bash
|
||||
rm -rf src/JdeScoping.Core/Models/Pipelines/
|
||||
```
|
||||
|
||||
**Step 2: Delete IPipelineApiClient.cs**
|
||||
|
||||
```bash
|
||||
rm src/JdeScoping.Core/ApiContracts/IPipelineApiClient.cs
|
||||
```
|
||||
|
||||
**Step 3: Verify deletions**
|
||||
|
||||
```bash
|
||||
ls src/JdeScoping.Core/Models/Pipelines/ 2>/dev/null || echo "Pipelines folder deleted"
|
||||
ls src/JdeScoping.Core/ApiContracts/IPipelineApiClient.cs 2>/dev/null || echo "Interface deleted"
|
||||
```
|
||||
|
||||
Expected: Both checks confirm deletion.
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Remove API Routes
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/JdeScoping.Core/ApiContracts/ApiRoutes.cs:155-188`
|
||||
|
||||
**Step 1: Remove the Pipelines class from ApiRoutes.cs**
|
||||
|
||||
Delete the entire `Pipelines` static class (lines 155-188):
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Routes for pipeline configuration API endpoints.
|
||||
/// </summary>
|
||||
public static class Pipelines
|
||||
{
|
||||
/// <summary>Base route for pipeline endpoints.</summary>
|
||||
public const string Base = "api/pipelines";
|
||||
|
||||
/// <summary>Route template for getting a pipeline by name.</summary>
|
||||
public const string ByName = "{name}";
|
||||
|
||||
/// <summary>Route template for getting pipeline status.</summary>
|
||||
public const string Status = "{name}/status";
|
||||
|
||||
/// <summary>Route template for getting pipeline executions.</summary>
|
||||
public const string Executions = "{name}/executions";
|
||||
|
||||
/// <summary>Builds the route to get a specific pipeline config.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetByName(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}";
|
||||
|
||||
/// <summary>Builds the route to get pipeline status.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetStatus(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}/status";
|
||||
|
||||
/// <summary>Builds the route to get pipeline executions.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <param name="count">The number of recent executions to retrieve.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetExecutions(string name, int count = 10) =>
|
||||
$"api/pipelines/{Uri.EscapeDataString(name)}/executions?count={count}";
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Verify the change**
|
||||
|
||||
```bash
|
||||
grep -n "Pipelines" src/JdeScoping.Core/ApiContracts/ApiRoutes.cs || echo "Pipelines routes removed"
|
||||
```
|
||||
|
||||
Expected: "Pipelines routes removed"
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Delete Plan Documents
|
||||
|
||||
**Files:**
|
||||
- Delete: `docs/plans/2026-01-07-pipeline-viewer-design.md`
|
||||
- Delete: `docs/plans/2026-01-07-pipeline-viewer-implementation.md`
|
||||
|
||||
**Step 1: Delete design document**
|
||||
|
||||
```bash
|
||||
rm docs/plans/2026-01-07-pipeline-viewer-design.md
|
||||
```
|
||||
|
||||
**Step 2: Delete implementation plan**
|
||||
|
||||
```bash
|
||||
rm docs/plans/2026-01-07-pipeline-viewer-implementation.md
|
||||
```
|
||||
|
||||
**Step 3: Verify deletions**
|
||||
|
||||
```bash
|
||||
ls docs/plans/*pipeline-viewer* 2>/dev/null || echo "Plan documents deleted"
|
||||
```
|
||||
|
||||
Expected: "Plan documents deleted"
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Build and Verify
|
||||
|
||||
**Step 1: Build the solution**
|
||||
|
||||
```bash
|
||||
cd /Users/dohertj2/Desktop/JdeScopingTool/NEW && dotnet build
|
||||
```
|
||||
|
||||
Expected: Build succeeds with no errors.
|
||||
|
||||
**Step 2: Search for any remaining pipeline viewer references**
|
||||
|
||||
```bash
|
||||
grep -r "PipelineViewer\|PipelineApiClient\|IPipelineApiClient\|PipelineController\|IPipelineMapper\|PipelineMapper\|pipeline-viewer" src/ --include="*.cs" --include="*.razor" || echo "No references remain"
|
||||
```
|
||||
|
||||
Expected: "No references remain"
|
||||
|
||||
**Step 3: Run tests**
|
||||
|
||||
```bash
|
||||
dotnet test
|
||||
```
|
||||
|
||||
Expected: All tests pass.
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Commit
|
||||
|
||||
**Step 1: Stage all changes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
```
|
||||
|
||||
**Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(webui): remove pipeline viewer feature
|
||||
|
||||
Remove the read-only pipeline viewer from the web UI:
|
||||
- Delete PipelineViewer.razor page and supporting components
|
||||
- Delete PipelineController and PipelineMapper from API
|
||||
- Delete Pipeline DTOs from Core
|
||||
- Delete PipelineApiClient from Client
|
||||
- Remove navigation link and DI registrations
|
||||
- Delete obsolete plan documents
|
||||
|
||||
The ConfigManager utility retains pipeline editing capabilities.
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Task | Action | Files Affected |
|
||||
|------|--------|----------------|
|
||||
| 1 | Delete Blazor components | 3 files deleted |
|
||||
| 2 | Remove nav link | 1 file modified |
|
||||
| 3 | Remove client service | 1 deleted, 1 modified |
|
||||
| 4 | Remove API layer | 3 deleted, 1 modified |
|
||||
| 5 | Remove Core DTOs | 5 files deleted |
|
||||
| 6 | Remove API routes | 1 file modified |
|
||||
| 7 | Delete plan docs | 2 files deleted |
|
||||
| 8 | Build and verify | - |
|
||||
| 9 | Commit | - |
|
||||
|
||||
**Total:** 14 files deleted, 4 files modified
|
||||
@@ -1,144 +0,0 @@
|
||||
using JdeScoping.Api.Mapping;
|
||||
using JdeScoping.Core.ApiContracts;
|
||||
using JdeScoping.Core.Models.Pipelines;
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
using JdeScoping.DataSync.Contracts;
|
||||
using JdeScoping.DataSync.Services;
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
|
||||
namespace JdeScoping.Api.Controllers;
|
||||
|
||||
/// <summary>
|
||||
/// API endpoints for pipeline configuration and status.
|
||||
/// </summary>
|
||||
[Route(ApiRoutes.Pipelines.Base)]
|
||||
[Authorize]
|
||||
public class PipelineController : ApiControllerBase
|
||||
{
|
||||
private readonly IEtlPipelineFactory _pipelineFactory;
|
||||
private readonly IDataUpdateRepository _dataUpdateRepository;
|
||||
private readonly IPipelineMapper _mapper;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="PipelineController"/> class.
|
||||
/// </summary>
|
||||
/// <param name="pipelineFactory">The ETL pipeline factory.</param>
|
||||
/// <param name="dataUpdateRepository">The data update repository.</param>
|
||||
/// <param name="mapper">The pipeline mapper.</param>
|
||||
public PipelineController(
|
||||
IEtlPipelineFactory pipelineFactory,
|
||||
IDataUpdateRepository dataUpdateRepository,
|
||||
IPipelineMapper mapper)
|
||||
{
|
||||
_pipelineFactory = pipelineFactory;
|
||||
_dataUpdateRepository = dataUpdateRepository;
|
||||
_mapper = mapper;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets list of all available pipeline names.
|
||||
/// </summary>
|
||||
[HttpGet]
|
||||
public ActionResult<PipelineListResponse> GetPipelineNames()
|
||||
{
|
||||
var names = _pipelineFactory.GetAvailableTables()
|
||||
.OrderBy(n => n)
|
||||
.ToList();
|
||||
return Ok(new PipelineListResponse(names));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets configuration for a specific pipeline.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
[HttpGet(ApiRoutes.Pipelines.ByName)]
|
||||
public ActionResult<PipelineConfigDto> GetPipeline(string name)
|
||||
{
|
||||
var config = _pipelineFactory.GetPipelineConfig(name);
|
||||
if (config is null)
|
||||
return NotFound();
|
||||
|
||||
var defaults = _pipelineFactory.GetScheduleDefaults();
|
||||
var dto = _mapper.MapToDto(name, config, defaults);
|
||||
return Ok(dto);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets schedule status for a pipeline.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="cancellationToken">The cancellation token.</param>
|
||||
[HttpGet(ApiRoutes.Pipelines.Status)]
|
||||
public async Task<ActionResult<PipelineStatusResponse>> GetStatus(
|
||||
string name,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var config = _pipelineFactory.GetPipelineConfig(name);
|
||||
if (config is null)
|
||||
return NotFound();
|
||||
|
||||
var tableName = config.Destination.Table;
|
||||
var lastRuns = await _dataUpdateRepository.GetLastRunsAsync(tableName, cancellationToken);
|
||||
var lastSuccessful = await _dataUpdateRepository.GetLastDataUpdatesAsync(cancellationToken);
|
||||
var defaults = _pipelineFactory.GetScheduleDefaults();
|
||||
|
||||
var statuses = new List<PipelineScheduleStatusDto>();
|
||||
foreach (var updateType in new[] { UpdateTypes.Mass, UpdateTypes.Daily, UpdateTypes.Hourly })
|
||||
{
|
||||
var scheduleConfig = _mapper.GetScheduleConfig(config, updateType);
|
||||
var interval = _mapper.GetEffectiveInterval(scheduleConfig, defaults, updateType);
|
||||
|
||||
lastRuns.TryGetValue(updateType, out var lastRun);
|
||||
var successKey = $"{tableName}_{(int)updateType}";
|
||||
lastSuccessful.TryGetValue(successKey, out var lastSuccess);
|
||||
|
||||
var nextRequired = lastSuccess?.EndDt?.AddMinutes(interval);
|
||||
var isOverdue = DataUpdateRepository.IsOverdue(
|
||||
lastSuccess?.EndDt, tableName, updateType, null);
|
||||
|
||||
statuses.Add(new PipelineScheduleStatusDto(
|
||||
updateType,
|
||||
lastRun?.StartDt,
|
||||
lastRun?.WasSuccessful ?? false,
|
||||
lastSuccess?.EndDt,
|
||||
nextRequired,
|
||||
isOverdue,
|
||||
interval));
|
||||
}
|
||||
|
||||
return Ok(new PipelineStatusResponse(statuses));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets recent execution history for a pipeline.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="count">The maximum number of recent executions to retrieve.</param>
|
||||
/// <param name="cancellationToken">The cancellation token.</param>
|
||||
[HttpGet(ApiRoutes.Pipelines.Executions)]
|
||||
public async Task<ActionResult<PipelineExecutionsResponse>> GetExecutions(
|
||||
string name,
|
||||
[FromQuery] int count = 30,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
var config = _pipelineFactory.GetPipelineConfig(name);
|
||||
if (config is null)
|
||||
return NotFound();
|
||||
|
||||
var tableName = config.Destination.Table;
|
||||
var updates = await _dataUpdateRepository.GetRecentUpdatesAsync(
|
||||
tableName, null, count, cancellationToken);
|
||||
|
||||
var executions = updates.Select(u => new PipelineExecutionDto(
|
||||
u.UpdateType,
|
||||
u.StartDt,
|
||||
u.EndDt == default ? null : u.EndDt,
|
||||
u.EndDt == default ? null : u.EndDt - u.StartDt,
|
||||
u.NumberRecords,
|
||||
u.WasSuccessful
|
||||
)).ToList();
|
||||
|
||||
return Ok(new PipelineExecutionsResponse(executions));
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,5 @@
|
||||
using System.Text.Json.Serialization;
|
||||
using JdeScoping.Api.Hubs;
|
||||
using JdeScoping.Api.Mapping;
|
||||
using JdeScoping.Api.Options;
|
||||
using JdeScoping.Api.Services;
|
||||
using JdeScoping.Core.Interfaces;
|
||||
@@ -39,9 +38,6 @@ public static class ApiDependencyInjection
|
||||
// Register TimeProvider for testability (allows mocking DateTime.UtcNow)
|
||||
services.AddSingleton(TimeProvider.System);
|
||||
|
||||
// Register mappers
|
||||
services.AddSingleton<IPipelineMapper, PipelineMapper>();
|
||||
|
||||
// Configure SignalR
|
||||
services.AddSignalR();
|
||||
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
using JdeScoping.Core.Models.Pipelines;
|
||||
using JdeScoping.DataSync.Configuration;
|
||||
|
||||
namespace JdeScoping.Api.Mapping;
|
||||
|
||||
/// <summary>
|
||||
/// Mapper for pipeline configuration to DTOs.
|
||||
/// </summary>
|
||||
public interface IPipelineMapper
|
||||
{
|
||||
/// <summary>
|
||||
/// Maps a pipeline configuration to its DTO representation.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="config">The pipeline configuration.</param>
|
||||
/// <param name="defaults">The default schedule settings.</param>
|
||||
PipelineConfigDto MapToDto(string name, PipelineConfig config, ScheduleDefaults defaults);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the effective interval for a schedule, applying defaults if not specified.
|
||||
/// </summary>
|
||||
/// <param name="config">The schedule configuration, or null to use defaults.</param>
|
||||
/// <param name="defaults">The default schedule settings.</param>
|
||||
/// <param name="updateType">The type of update to get the interval for.</param>
|
||||
int GetEffectiveInterval(ScheduleConfig? config, ScheduleDefaults defaults, UpdateTypes updateType);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the schedule configuration for a specific update type.
|
||||
/// </summary>
|
||||
/// <param name="config">The pipeline configuration.</param>
|
||||
/// <param name="updateType">The type of update to get the configuration for.</param>
|
||||
ScheduleConfig? GetScheduleConfig(PipelineConfig config, UpdateTypes updateType);
|
||||
}
|
||||
@@ -1,108 +0,0 @@
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
using JdeScoping.Core.Models.Pipelines;
|
||||
using JdeScoping.DataSync.Configuration;
|
||||
|
||||
namespace JdeScoping.Api.Mapping;
|
||||
|
||||
/// <summary>
|
||||
/// Maps pipeline configuration to DTOs.
|
||||
/// </summary>
|
||||
public class PipelineMapper : IPipelineMapper
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public PipelineConfigDto MapToDto(
|
||||
string name,
|
||||
PipelineConfig config,
|
||||
ScheduleDefaults defaults)
|
||||
{
|
||||
var source = new PipelineSourceDto(
|
||||
config.Source.Connection,
|
||||
Truncate(config.Source.Query),
|
||||
Truncate(config.Source.MassQuery),
|
||||
config.Source.Query,
|
||||
config.Source.MassQuery,
|
||||
config.Source.Parameters?.Select(p => new PipelineParameterDto(
|
||||
p.Key, p.Value.Format, p.Value.Source)).ToList() ?? []);
|
||||
|
||||
var matchCols = config.Destination.MatchColumns?.ToList();
|
||||
var destination = new PipelineDestinationDto(
|
||||
config.Destination.Table,
|
||||
matchCols?.Count > 0 ? "BulkMerge" : "BulkImport",
|
||||
matchCols,
|
||||
config.Destination.ExcludeFromUpdate?.ToList());
|
||||
|
||||
// Mass uses massQuery with no parameters; Daily/Hourly use query with parameters
|
||||
var parameters = config.Source.Parameters?.Select(p => new PipelineParameterDto(
|
||||
p.Key, p.Value.Format, p.Value.Source)).ToList() ?? [];
|
||||
|
||||
var schedules = new PipelineSchedulesDto(
|
||||
MapSchedule(config.Schedules?.Mass, defaults.Mass, config.Source.MassQuery, [], config.PreScripts, config.PostScripts),
|
||||
MapSchedule(config.Schedules?.Daily, defaults.Daily, config.Source.Query, parameters, config.PreScripts, config.PostScripts),
|
||||
MapSchedule(config.Schedules?.Hourly, defaults.Hourly, config.Source.Query, parameters, config.PreScripts, config.PostScripts));
|
||||
|
||||
return new PipelineConfigDto(
|
||||
name,
|
||||
source,
|
||||
destination,
|
||||
schedules,
|
||||
config.PreScripts?.Count ?? 0,
|
||||
config.PostScripts?.Count ?? 0,
|
||||
config.PreScripts,
|
||||
config.PostScripts);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public ScheduleConfig? GetScheduleConfig(
|
||||
PipelineConfig config,
|
||||
UpdateTypes updateType) => updateType switch
|
||||
{
|
||||
UpdateTypes.Mass => config.Schedules?.Mass,
|
||||
UpdateTypes.Daily => config.Schedules?.Daily,
|
||||
UpdateTypes.Hourly => config.Schedules?.Hourly,
|
||||
_ => null
|
||||
};
|
||||
|
||||
/// <inheritdoc />
|
||||
public int GetEffectiveInterval(
|
||||
ScheduleConfig? config,
|
||||
ScheduleDefaults defaults,
|
||||
UpdateTypes updateType)
|
||||
{
|
||||
if (config?.IntervalMinutes > 0)
|
||||
return config.IntervalMinutes;
|
||||
|
||||
return updateType switch
|
||||
{
|
||||
UpdateTypes.Mass => defaults.Mass.IntervalMinutes,
|
||||
UpdateTypes.Daily => defaults.Daily.IntervalMinutes,
|
||||
UpdateTypes.Hourly => defaults.Hourly.IntervalMinutes,
|
||||
_ => 60
|
||||
};
|
||||
}
|
||||
|
||||
private static PipelineScheduleDto MapSchedule(
|
||||
ScheduleConfig? config,
|
||||
ScheduleConfig defaults,
|
||||
string? query,
|
||||
List<PipelineParameterDto> parameters,
|
||||
List<string>? preScripts,
|
||||
List<string>? postScripts)
|
||||
{
|
||||
return new PipelineScheduleDto(
|
||||
config?.Enabled ?? defaults.Enabled,
|
||||
config?.IntervalMinutes > 0 ? config.IntervalMinutes : defaults.IntervalMinutes,
|
||||
config?.PrePurge ?? defaults.PrePurge,
|
||||
config?.ReIndex ?? defaults.ReIndex,
|
||||
config?.IntervalMinutes > 0 && config.IntervalMinutes != defaults.IntervalMinutes,
|
||||
config?.PrePurge != null && config.PrePurge != defaults.PrePurge,
|
||||
config?.ReIndex != null && config.ReIndex != defaults.ReIndex,
|
||||
query,
|
||||
parameters,
|
||||
preScripts,
|
||||
postScripts);
|
||||
}
|
||||
|
||||
private static string? Truncate(string? value, int maxLength = 100) =>
|
||||
value is null ? null :
|
||||
value.Length <= maxLength ? value : value[..maxLength] + "...";
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
@namespace JdeScoping.Client.Components.Admin
|
||||
@using JdeScoping.Core.Models.Pipelines
|
||||
@using JdeScoping.Core.Models.Enums
|
||||
@using JdeScoping.Client.Helpers
|
||||
|
||||
<RadzenCard class="rz-mb-4">
|
||||
<RadzenRow AlignItems="AlignItems.Center" class="rz-mb-3">
|
||||
<RadzenColumn>
|
||||
<RadzenText TextStyle="TextStyle.H5" class="rz-m-0">
|
||||
@GetScheduleTypeName(ScheduleType) Refresh
|
||||
</RadzenText>
|
||||
</RadzenColumn>
|
||||
<RadzenColumn Size="2" class="rz-text-align-end">
|
||||
@if (Config?.Enabled == true)
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="Enabled" />
|
||||
}
|
||||
else
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Light" Text="Disabled" />
|
||||
}
|
||||
</RadzenColumn>
|
||||
</RadzenRow>
|
||||
|
||||
@if (Config is not null)
|
||||
{
|
||||
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mb-2">Schedule Settings</RadzenText>
|
||||
<table class="rz-mb-4" style="width: 100%; border-collapse: collapse;">
|
||||
<thead>
|
||||
<tr style="border-bottom: 1px solid #dee2e6;">
|
||||
<th style="text-align: left; padding: 0.5rem;">Setting</th>
|
||||
<th style="text-align: left; padding: 0.5rem;">Value</th>
|
||||
<th style="text-align: left; padding: 0.5rem;">Source</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr style="border-bottom: 1px solid #dee2e6;">
|
||||
<td style="padding: 0.5rem;">Interval</td>
|
||||
<td style="padding: 0.5rem;">@FormatInterval(Config.IntervalMinutes)</td>
|
||||
<td style="padding: 0.5rem;">@(Config.IntervalIsOverride ? "Override" : "Default")</td>
|
||||
</tr>
|
||||
<tr style="border-bottom: 1px solid #dee2e6;">
|
||||
<td style="padding: 0.5rem;">Pre-Purge</td>
|
||||
<td style="padding: 0.5rem;">@(Config.PrePurge ? "Yes" : "No")</td>
|
||||
<td style="padding: 0.5rem;">@(Config.PrePurgeIsOverride ? "Override" : "Default")</td>
|
||||
</tr>
|
||||
<tr style="border-bottom: 1px solid #dee2e6;">
|
||||
<td style="padding: 0.5rem;">Re-Index</td>
|
||||
<td style="padding: 0.5rem;">@(Config.ReIndex ? "Yes" : "No")</td>
|
||||
<td style="padding: 0.5rem;">@(Config.ReIndexIsOverride ? "Override" : "Default")</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
@if (Config.Parameters?.Count > 0)
|
||||
{
|
||||
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mb-2">Parameters</RadzenText>
|
||||
<ul class="rz-mb-4">
|
||||
@foreach (var param in Config.Parameters)
|
||||
{
|
||||
<li><strong>@param.Name</strong>: @(param.Format ?? "default") (source: @param.Source)</li>
|
||||
}
|
||||
</ul>
|
||||
}
|
||||
|
||||
@if (!string.IsNullOrWhiteSpace(Config.Query))
|
||||
{
|
||||
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mb-2">Query</RadzenText>
|
||||
<pre style="background: #f8f9fa; padding: 1rem; border-radius: 4px; font-family: 'Consolas', 'Monaco', 'Courier New', monospace; font-size: 0.875rem; line-height: 1.5; overflow-x: auto; white-space: pre-wrap; word-break: break-word; max-height: 400px; overflow-y: auto;">@SqlFormatHelper.FormatSql(Config.Query)</pre>
|
||||
}
|
||||
|
||||
@if (Config.PreScripts?.Count > 0)
|
||||
{
|
||||
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mt-4 rz-mb-2">Pre-Scripts (@Config.PreScripts.Count)</RadzenText>
|
||||
@for (int i = 0; i < Config.PreScripts.Count; i++)
|
||||
{
|
||||
var script = Config.PreScripts[i];
|
||||
<RadzenText TextStyle="TextStyle.Body2" class="rz-mb-1"><strong>Script @(i + 1):</strong></RadzenText>
|
||||
<pre style="background: #fff3cd; padding: 0.75rem; border-radius: 4px; font-family: 'Consolas', 'Monaco', 'Courier New', monospace; font-size: 0.8rem; line-height: 1.4; overflow-x: auto; white-space: pre-wrap; word-break: break-word; max-height: 200px; overflow-y: auto; margin-bottom: 0.5rem;">@SqlFormatHelper.FormatSql(script)</pre>
|
||||
}
|
||||
}
|
||||
|
||||
@if (Config.PostScripts?.Count > 0)
|
||||
{
|
||||
<RadzenText TextStyle="TextStyle.Subtitle1" class="rz-mt-4 rz-mb-2">Post-Scripts (@Config.PostScripts.Count)</RadzenText>
|
||||
@for (int i = 0; i < Config.PostScripts.Count; i++)
|
||||
{
|
||||
var script = Config.PostScripts[i];
|
||||
<RadzenText TextStyle="TextStyle.Body2" class="rz-mb-1"><strong>Script @(i + 1):</strong></RadzenText>
|
||||
<pre style="background: #d1ecf1; padding: 0.75rem; border-radius: 4px; font-family: 'Consolas', 'Monaco', 'Courier New', monospace; font-size: 0.8rem; line-height: 1.4; overflow-x: auto; white-space: pre-wrap; word-break: break-word; max-height: 200px; overflow-y: auto; margin-bottom: 0.5rem;">@SqlFormatHelper.FormatSql(script)</pre>
|
||||
}
|
||||
}
|
||||
}
|
||||
</RadzenCard>
|
||||
|
||||
@code {
|
||||
[Parameter] public UpdateTypes ScheduleType { get; set; }
|
||||
[Parameter] public PipelineScheduleDto? Config { get; set; }
|
||||
|
||||
private static string GetScheduleTypeName(UpdateTypes type) => type switch
|
||||
{
|
||||
UpdateTypes.Mass => "Mass",
|
||||
UpdateTypes.Daily => "Daily",
|
||||
UpdateTypes.Hourly => "Hourly",
|
||||
_ => type.ToString()
|
||||
};
|
||||
|
||||
private static string FormatInterval(int minutes)
|
||||
{
|
||||
if (minutes >= 1440)
|
||||
return $"{minutes / 1440} day(s) ({minutes} min)";
|
||||
if (minutes >= 60)
|
||||
return $"{minutes / 60} hour(s) ({minutes} min)";
|
||||
return $"{minutes} minutes";
|
||||
}
|
||||
}
|
||||
@@ -1,101 +0,0 @@
|
||||
@namespace JdeScoping.Client.Components.Admin
|
||||
@using JdeScoping.Client.Helpers
|
||||
@inject IJSRuntime JS
|
||||
|
||||
@if (Visible)
|
||||
{
|
||||
<div class="sql-modal-overlay" @onclick="Close">
|
||||
<div class="sql-modal-content" @onclick:stopPropagation="true">
|
||||
<div class="sql-modal-header">
|
||||
<RadzenText TextStyle="TextStyle.H5" class="rz-m-0">@Title</RadzenText>
|
||||
<RadzenButton Icon="close" ButtonStyle="ButtonStyle.Light" Size="ButtonSize.Small" Click="Close" />
|
||||
</div>
|
||||
<div class="sql-modal-body">
|
||||
<pre>@FormattedSql</pre>
|
||||
</div>
|
||||
<div class="sql-modal-footer">
|
||||
<RadzenButton Text="Copy to Clipboard" Icon="content_copy" ButtonStyle="ButtonStyle.Secondary" Click="CopyToClipboard" class="rz-mr-2" />
|
||||
<RadzenButton Text="Close" Click="Close" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
}
|
||||
|
||||
<style>
|
||||
.sql-modal-overlay {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background-color: rgba(0, 0, 0, 0.5);
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
z-index: 1000;
|
||||
}
|
||||
|
||||
.sql-modal-content {
|
||||
background: white;
|
||||
border-radius: 8px;
|
||||
width: 80vw;
|
||||
max-width: 1200px;
|
||||
max-height: 80vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
box-shadow: 0 4px 20px rgba(0, 0, 0, 0.2);
|
||||
}
|
||||
|
||||
.sql-modal-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 1rem 1.5rem;
|
||||
border-bottom: 1px solid #dee2e6;
|
||||
}
|
||||
|
||||
.sql-modal-body {
|
||||
flex: 1;
|
||||
overflow: auto;
|
||||
padding: 1.5rem;
|
||||
background: #f8f9fa;
|
||||
}
|
||||
|
||||
.sql-modal-body pre {
|
||||
margin: 0;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
font-family: 'Consolas', 'Monaco', 'Courier New', monospace;
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.sql-modal-footer {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
padding: 1rem 1.5rem;
|
||||
border-top: 1px solid #dee2e6;
|
||||
}
|
||||
</style>
|
||||
|
||||
@code {
|
||||
[Parameter] public bool Visible { get; set; }
|
||||
[Parameter] public EventCallback<bool> VisibleChanged { get; set; }
|
||||
[Parameter] public string? Title { get; set; }
|
||||
[Parameter] public string? Sql { get; set; }
|
||||
|
||||
private string FormattedSql => SqlFormatHelper.FormatSql(Sql);
|
||||
|
||||
private async Task CopyToClipboard()
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(Sql))
|
||||
{
|
||||
await JS.InvokeVoidAsync("navigator.clipboard.writeText", Sql);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task Close()
|
||||
{
|
||||
await VisibleChanged.InvokeAsync(false);
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,6 @@
|
||||
<NavLink class="nav-link" href="/search">New Search</NavLink>
|
||||
<NavLink class="nav-link" href="/search/queue">Search Queue</NavLink>
|
||||
<NavLink class="nav-link" href="/refresh-status">Refresh Status</NavLink>
|
||||
<NavLink class="nav-link" href="/admin/pipeline-viewer">Pipeline Viewer</NavLink>
|
||||
</nav>
|
||||
</div>
|
||||
<div class="navbar-right">
|
||||
|
||||
@@ -1,243 +0,0 @@
|
||||
@*
|
||||
PipelineViewer.razor - ETL pipeline configuration viewer (admin).
|
||||
|
||||
Displays all configured data sync pipelines with their schedules, queries, and mappings.
|
||||
Read-only view for inspecting pipeline configuration without modifying.
|
||||
*@
|
||||
@page "/admin/pipeline-viewer"
|
||||
@attribute [Authorize]
|
||||
@using JdeScoping.Core.ApiContracts
|
||||
@using JdeScoping.Core.Models.Pipelines
|
||||
@using JdeScoping.Core.Models.Enums
|
||||
@inject IPipelineApiClient PipelineApi
|
||||
|
||||
<PageTitle>Pipeline Configuration Viewer - JDE Scoping Tool</PageTitle>
|
||||
|
||||
<RadzenText TextStyle="TextStyle.H4" class="rz-mb-4">ETL Pipeline Configuration Viewer</RadzenText>
|
||||
|
||||
<!-- Pipeline Selector -->
|
||||
<RadzenCard class="rz-mb-4">
|
||||
<RadzenFormField Text="Select Pipeline" Style="width: 400px;">
|
||||
<RadzenDropDown @bind-Value="_selectedPipeline" Data="@_pipelineNames" Style="width: 100%;"
|
||||
Change="@OnPipelineChanged" Placeholder="Select a pipeline..." />
|
||||
</RadzenFormField>
|
||||
</RadzenCard>
|
||||
|
||||
@if (_isLoading)
|
||||
{
|
||||
<LoadingIndicator Message="Loading pipeline data..." />
|
||||
}
|
||||
else if (_config is not null)
|
||||
{
|
||||
<!-- Status Summary Table -->
|
||||
<RadzenCard class="rz-mb-4">
|
||||
<RadzenText TextStyle="TextStyle.H5" class="rz-mb-3">Schedule Status Summary</RadzenText>
|
||||
<RadzenDataGrid Data="@_statuses" TItem="PipelineScheduleStatusDto">
|
||||
<Columns>
|
||||
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Property="ScheduleType" Title="Type" Width="100px">
|
||||
<Template Context="item">@item.ScheduleType.ToString()</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Last Run" Width="160px">
|
||||
<Template Context="item">@(item.LastRun?.ToString("MM/dd/yyyy hh:mm tt") ?? "Never")</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Success?" Width="80px" TextAlign="TextAlign.Center">
|
||||
<Template Context="item">
|
||||
@if (item.LastRun.HasValue)
|
||||
{
|
||||
@if (item.LastRunWasSuccessful)
|
||||
{
|
||||
<RadzenIcon Icon="check_circle" Style="color: green;" />
|
||||
}
|
||||
else
|
||||
{
|
||||
<RadzenIcon Icon="cancel" Style="color: red;" />
|
||||
}
|
||||
}
|
||||
</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Next Required" Width="160px">
|
||||
<Template Context="item">@(item.NextRequiredRun?.ToString("MM/dd/yyyy hh:mm tt") ?? "N/A")</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineScheduleStatusDto" Title="Status" Width="100px" TextAlign="TextAlign.Center">
|
||||
<Template Context="item">
|
||||
@if (item.IsOverdue)
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Warning" Text="Overdue" />
|
||||
}
|
||||
else
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="OK" />
|
||||
}
|
||||
</Template>
|
||||
</RadzenDataGridColumn>
|
||||
</Columns>
|
||||
</RadzenDataGrid>
|
||||
</RadzenCard>
|
||||
|
||||
<!-- Execution History Table -->
|
||||
<RadzenCard class="rz-mb-4">
|
||||
<RadzenText TextStyle="TextStyle.H5" class="rz-mb-3">Recent Execution History</RadzenText>
|
||||
<RadzenDataGrid Data="@_executions" TItem="PipelineExecutionDto" AllowSorting="true" PageSize="10" AllowPaging="true">
|
||||
<Columns>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Property="ScheduleType" Title="Type" Width="100px">
|
||||
<Template Context="item">@item.ScheduleType.ToString()</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Start" Width="160px">
|
||||
<Template Context="item">@item.StartTime.ToString("MM/dd/yyyy hh:mm tt")</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="End" Width="160px">
|
||||
<Template Context="item">@(item.EndTime?.ToString("MM/dd/yyyy hh:mm tt") ?? "In Progress")</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Duration" Width="100px">
|
||||
<Template Context="item">@FormatDuration(item.Duration)</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Property="RecordCount" Title="Records" Width="100px" TextAlign="TextAlign.Right">
|
||||
<Template Context="item">@item.RecordCount.ToString("N0")</Template>
|
||||
</RadzenDataGridColumn>
|
||||
<RadzenDataGridColumn TItem="PipelineExecutionDto" Title="Result" Width="100px" TextAlign="TextAlign.Center">
|
||||
<Template Context="item">
|
||||
@if (item.WasSuccessful)
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="Success" />
|
||||
}
|
||||
else
|
||||
{
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Danger" Text="Failed" />
|
||||
}
|
||||
</Template>
|
||||
</RadzenDataGridColumn>
|
||||
</Columns>
|
||||
</RadzenDataGrid>
|
||||
</RadzenCard>
|
||||
|
||||
<!-- Common Pipeline Info -->
|
||||
<RadzenRow Gap="1rem" class="rz-mb-4">
|
||||
<!-- Source Card -->
|
||||
<RadzenColumn Size="6">
|
||||
<RadzenCard Style="height: 100%;">
|
||||
<RadzenText TextStyle="TextStyle.H6" class="rz-mb-2">Source</RadzenText>
|
||||
<p><strong>Connection:</strong>
|
||||
@switch (_config.Source.Connection.ToLower())
|
||||
{
|
||||
case "jde":
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Info" Text="JDE" />
|
||||
break;
|
||||
case "cms":
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Success" Text="CMS" />
|
||||
break;
|
||||
case "giw":
|
||||
<RadzenBadge BadgeStyle="BadgeStyle.Warning" Text="GIW" />
|
||||
break;
|
||||
default:
|
||||
<RadzenBadge Text="@_config.Source.Connection" />
|
||||
break;
|
||||
}
|
||||
</p>
|
||||
</RadzenCard>
|
||||
</RadzenColumn>
|
||||
|
||||
<!-- Destination Card -->
|
||||
<RadzenColumn Size="6">
|
||||
<RadzenCard Style="height: 100%;">
|
||||
<RadzenText TextStyle="TextStyle.H6" class="rz-mb-2">Destination</RadzenText>
|
||||
<p><strong>Table:</strong> @_config.Destination.Table</p>
|
||||
<p><strong>Operation:</strong>
|
||||
<RadzenBadge BadgeStyle="@(_config.Destination.OperationType == "BulkMerge" ? BadgeStyle.Primary : BadgeStyle.Secondary)"
|
||||
Text="@_config.Destination.OperationType" />
|
||||
</p>
|
||||
@if (_config.Destination.MatchColumns?.Count > 0)
|
||||
{
|
||||
<p><strong>Match Columns:</strong></p>
|
||||
<ul>
|
||||
@foreach (var col in _config.Destination.MatchColumns)
|
||||
{
|
||||
<li>@col</li>
|
||||
}
|
||||
</ul>
|
||||
}
|
||||
@if (_config.Destination.ExcludeFromUpdate?.Count > 0)
|
||||
{
|
||||
<p><strong>Exclude:</strong></p>
|
||||
<ul>
|
||||
@foreach (var col in _config.Destination.ExcludeFromUpdate)
|
||||
{
|
||||
<li>@col</li>
|
||||
}
|
||||
</ul>
|
||||
}
|
||||
</RadzenCard>
|
||||
</RadzenColumn>
|
||||
</RadzenRow>
|
||||
|
||||
<!-- Schedule Sections -->
|
||||
<PipelineScheduleSection ScheduleType="UpdateTypes.Mass" Config="@_config.Schedules.Mass" />
|
||||
|
||||
<PipelineScheduleSection ScheduleType="UpdateTypes.Daily" Config="@_config.Schedules.Daily" />
|
||||
|
||||
<PipelineScheduleSection ScheduleType="UpdateTypes.Hourly" Config="@_config.Schedules.Hourly" />
|
||||
}
|
||||
|
||||
@code {
|
||||
private List<string> _pipelineNames = [];
|
||||
private string? _selectedPipeline;
|
||||
private bool _isLoading;
|
||||
private PipelineConfigDto? _config;
|
||||
private List<PipelineScheduleStatusDto> _statuses = [];
|
||||
private List<PipelineExecutionDto> _executions = [];
|
||||
|
||||
protected override async Task OnInitializedAsync()
|
||||
{
|
||||
var result = await PipelineApi.GetPipelineNamesAsync();
|
||||
if (result.IsSuccess)
|
||||
{
|
||||
_pipelineNames = result.Value.PipelineNames;
|
||||
}
|
||||
}
|
||||
|
||||
private async Task OnPipelineChanged()
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(_selectedPipeline))
|
||||
{
|
||||
_config = null;
|
||||
_statuses = [];
|
||||
_executions = [];
|
||||
return;
|
||||
}
|
||||
|
||||
_isLoading = true;
|
||||
try
|
||||
{
|
||||
var configTask = PipelineApi.GetPipelineAsync(_selectedPipeline);
|
||||
var statusTask = PipelineApi.GetStatusAsync(_selectedPipeline);
|
||||
var executionsTask = PipelineApi.GetExecutionsAsync(_selectedPipeline);
|
||||
|
||||
await Task.WhenAll(configTask, statusTask, executionsTask);
|
||||
|
||||
var configResult = configTask.Result;
|
||||
var statusResult = statusTask.Result;
|
||||
var executionsResult = executionsTask.Result;
|
||||
|
||||
if (configResult.IsSuccess)
|
||||
_config = configResult.Value;
|
||||
|
||||
if (statusResult.IsSuccess)
|
||||
_statuses = statusResult.Value.Statuses;
|
||||
|
||||
if (executionsResult.IsSuccess)
|
||||
_executions = executionsResult.Value.Executions;
|
||||
}
|
||||
finally
|
||||
{
|
||||
_isLoading = false;
|
||||
}
|
||||
}
|
||||
|
||||
private static string FormatDuration(TimeSpan? duration)
|
||||
{
|
||||
if (!duration.HasValue) return "-";
|
||||
var d = duration.Value;
|
||||
if (d.TotalHours >= 1) return $"{d.Hours}h {d.Minutes}m";
|
||||
if (d.TotalMinutes >= 1) return $"{d.Minutes}m {d.Seconds}s";
|
||||
return $"{d.Seconds}s";
|
||||
}
|
||||
}
|
||||
@@ -55,7 +55,6 @@ builder.Services.AddScoped<ISearchApiClient, SearchApiClient>();
|
||||
builder.Services.AddScoped<ILookupApiClient, LookupApiClient>();
|
||||
builder.Services.AddScoped<IAuthApiClient, AuthApiClient>();
|
||||
builder.Services.AddScoped<IFileApiClient, FileApiClient>();
|
||||
builder.Services.AddScoped<IPipelineApiClient, PipelineApiClient>();
|
||||
|
||||
// Search services
|
||||
builder.Services.AddScoped<ISearchValidationService, SearchValidationService>();
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
using JdeScoping.Core.ApiContracts;
|
||||
using JdeScoping.Core.Models.Pipelines;
|
||||
using JdeScoping.Core.ApiContracts.Results;
|
||||
|
||||
namespace JdeScoping.Client.Services;
|
||||
|
||||
/// <summary>
|
||||
/// HTTP client implementation for pipeline configuration API.
|
||||
/// </summary>
|
||||
public class PipelineApiClient : ApiClientBase, IPipelineApiClient
|
||||
{
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="PipelineApiClient"/> class.
|
||||
/// </summary>
|
||||
/// <param name="httpClient">The HTTP client for API communication.</param>
|
||||
public PipelineApiClient(HttpClient httpClient) : base(httpClient) { }
|
||||
|
||||
/// <summary>
|
||||
/// Gets the list of available pipeline names from the API.
|
||||
/// </summary>
|
||||
/// <param name="ct">Cancellation token for the request.</param>
|
||||
/// <returns>The API result containing the pipeline list.</returns>
|
||||
public Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default)
|
||||
=> GetAsync<PipelineListResponse>(ApiRoutes.Pipelines.Base, ct);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the configuration for a specific pipeline by name.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="ct">Cancellation token for the request.</param>
|
||||
/// <returns>The API result containing the pipeline configuration.</returns>
|
||||
public Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineConfigDto>(ApiRoutes.Pipelines.GetByName(name), ct);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the current execution status of a pipeline.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="ct">Cancellation token for the request.</param>
|
||||
/// <returns>The API result containing the pipeline status.</returns>
|
||||
public Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineStatusResponse>(ApiRoutes.Pipelines.GetStatus(name), ct);
|
||||
|
||||
/// <summary>
|
||||
/// Gets the recent execution history for a pipeline.
|
||||
/// </summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="count">The maximum number of execution records to retrieve.</param>
|
||||
/// <param name="ct">Cancellation token for the request.</param>
|
||||
/// <returns>The API result containing the execution history.</returns>
|
||||
public Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 30, CancellationToken ct = default)
|
||||
=> GetAsync<PipelineExecutionsResponse>(ApiRoutes.Pipelines.GetExecutions(name, count), ct);
|
||||
}
|
||||
@@ -13,7 +13,6 @@
|
||||
@using Radzen.Blazor
|
||||
@using JdeScoping.Client
|
||||
@using JdeScoping.Client.Auth
|
||||
@using JdeScoping.Client.Components.Admin
|
||||
@using JdeScoping.Client.Components.FilterPanels
|
||||
@using JdeScoping.Client.Extensions
|
||||
@using JdeScoping.Client.Components.Search
|
||||
@@ -25,6 +24,5 @@
|
||||
@using JdeScoping.Core.ApiContracts
|
||||
@using JdeScoping.Core.Models.Auth
|
||||
@using JdeScoping.Core.Models.Infrastructure
|
||||
@using JdeScoping.Core.Models.Pipelines
|
||||
@using JdeScoping.Core.ViewModels
|
||||
@using ClientSearchViewModel = JdeScoping.Client.Models.SearchViewModel
|
||||
|
||||
@@ -152,38 +152,4 @@ public static class ApiRoutes
|
||||
$"api/refresh-status?minDT={minDt:yyyy-MM-dd}&maxDT={maxDt:yyyy-MM-dd}";
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Routes for pipeline configuration API endpoints.
|
||||
/// </summary>
|
||||
public static class Pipelines
|
||||
{
|
||||
/// <summary>Base route for pipeline endpoints.</summary>
|
||||
public const string Base = "api/pipelines";
|
||||
|
||||
/// <summary>Route template for getting a pipeline by name.</summary>
|
||||
public const string ByName = "{name}";
|
||||
|
||||
/// <summary>Route template for getting pipeline status.</summary>
|
||||
public const string Status = "{name}/status";
|
||||
|
||||
/// <summary>Route template for getting pipeline executions.</summary>
|
||||
public const string Executions = "{name}/executions";
|
||||
|
||||
/// <summary>Builds the route to get a specific pipeline config.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetByName(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}";
|
||||
|
||||
/// <summary>Builds the route to get pipeline status.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetStatus(string name) => $"api/pipelines/{Uri.EscapeDataString(name)}/status";
|
||||
|
||||
/// <summary>Builds the route to get pipeline executions.</summary>
|
||||
/// <param name="name">The pipeline name to URL-encode.</param>
|
||||
/// <param name="count">The number of recent executions to retrieve.</param>
|
||||
/// <returns>The formatted route.</returns>
|
||||
public static string GetExecutions(string name, int count = 10) =>
|
||||
$"api/pipelines/{Uri.EscapeDataString(name)}/executions?count={count}";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
using JdeScoping.Core.Models.Pipelines;
|
||||
using JdeScoping.Core.ApiContracts.Results;
|
||||
|
||||
namespace JdeScoping.Core.ApiContracts;
|
||||
|
||||
/// <summary>
|
||||
/// Client contract for pipeline configuration API operations.
|
||||
/// </summary>
|
||||
public interface IPipelineApiClient
|
||||
{
|
||||
/// <summary>Gets list of all available pipeline names.</summary>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
Task<ApiResult<PipelineListResponse>> GetPipelineNamesAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>Gets configuration for a specific pipeline.</summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
Task<ApiResult<PipelineConfigDto>> GetPipelineAsync(string name, CancellationToken ct = default);
|
||||
|
||||
/// <summary>Gets schedule status for a pipeline.</summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
Task<ApiResult<PipelineStatusResponse>> GetStatusAsync(string name, CancellationToken ct = default);
|
||||
|
||||
/// <summary>Gets recent execution history for a pipeline.</summary>
|
||||
/// <param name="name">The pipeline name.</param>
|
||||
/// <param name="count">The maximum number of executions to return.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
Task<ApiResult<PipelineExecutionsResponse>> GetExecutionsAsync(string name, int count = 30, CancellationToken ct = default);
|
||||
}
|
||||
@@ -1,51 +0,0 @@
|
||||
namespace JdeScoping.Core.Models.Pipelines;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline configuration DTO for display.
|
||||
/// </summary>
|
||||
public record PipelineConfigDto(
|
||||
string Name,
|
||||
PipelineSourceDto Source,
|
||||
PipelineDestinationDto Destination,
|
||||
PipelineSchedulesDto Schedules,
|
||||
int PreScriptCount,
|
||||
int PostScriptCount,
|
||||
List<string>? PreScripts,
|
||||
List<string>? PostScripts);
|
||||
|
||||
public record PipelineSourceDto(
|
||||
string Connection,
|
||||
string? QueryPreview,
|
||||
string? MassQueryPreview,
|
||||
string? Query,
|
||||
string? MassQuery,
|
||||
List<PipelineParameterDto> Parameters);
|
||||
|
||||
public record PipelineParameterDto(
|
||||
string Name,
|
||||
string? Format,
|
||||
string Source);
|
||||
|
||||
public record PipelineDestinationDto(
|
||||
string Table,
|
||||
string OperationType,
|
||||
List<string>? MatchColumns,
|
||||
List<string>? ExcludeFromUpdate);
|
||||
|
||||
public record PipelineSchedulesDto(
|
||||
PipelineScheduleDto Mass,
|
||||
PipelineScheduleDto Daily,
|
||||
PipelineScheduleDto Hourly);
|
||||
|
||||
public record PipelineScheduleDto(
|
||||
bool Enabled,
|
||||
int IntervalMinutes,
|
||||
bool PrePurge,
|
||||
bool ReIndex,
|
||||
bool IntervalIsOverride,
|
||||
bool PrePurgeIsOverride,
|
||||
bool ReIndexIsOverride,
|
||||
string? Query,
|
||||
List<PipelineParameterDto> Parameters,
|
||||
List<string>? PreScripts,
|
||||
List<string>? PostScripts);
|
||||
@@ -1,16 +0,0 @@
|
||||
namespace JdeScoping.Core.Models.Pipelines;
|
||||
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline execution history.
|
||||
/// </summary>
|
||||
public record PipelineExecutionsResponse(List<PipelineExecutionDto> Executions);
|
||||
|
||||
public record PipelineExecutionDto(
|
||||
UpdateTypes ScheduleType,
|
||||
DateTime StartTime,
|
||||
DateTime? EndTime,
|
||||
TimeSpan? Duration,
|
||||
long RecordCount,
|
||||
bool WasSuccessful);
|
||||
@@ -1,6 +0,0 @@
|
||||
namespace JdeScoping.Core.Models.Pipelines;
|
||||
|
||||
/// <summary>
|
||||
/// Response containing list of available pipeline names.
|
||||
/// </summary>
|
||||
public record PipelineListResponse(List<string> PipelineNames);
|
||||
@@ -1,17 +0,0 @@
|
||||
namespace JdeScoping.Core.Models.Pipelines;
|
||||
|
||||
using JdeScoping.Core.Models.Enums;
|
||||
|
||||
/// <summary>
|
||||
/// Pipeline schedule status for each update type.
|
||||
/// </summary>
|
||||
public record PipelineStatusResponse(List<PipelineScheduleStatusDto> Statuses);
|
||||
|
||||
public record PipelineScheduleStatusDto(
|
||||
UpdateTypes ScheduleType,
|
||||
DateTime? LastRun,
|
||||
bool LastRunWasSuccessful,
|
||||
DateTime? LastSuccessfulRun,
|
||||
DateTime? NextRequiredRun,
|
||||
bool IsOverdue,
|
||||
int IntervalMinutes);
|
||||
Reference in New Issue
Block a user