08d2a07d8b
Replace JoeAppEngine tags with TestChildObject tags (TestBool, TestInt, TestFloat, TestDouble, TestString, TestDateTime, and array variants) in Phase 4 and Phase 7 plans. These tags cover all TypedValue oneof cases for comprehensive v2 testing.
667 lines
26 KiB
Markdown
667 lines
26 KiB
Markdown
# Phase 4: Host Health, Metrics & Status Server — Implementation Plan
|
|
|
|
**Date**: 2026-03-21
|
|
**Prerequisites**: Phase 3 complete and passing (gRPC server, Security, Configuration, Service Hosting all functional)
|
|
**Working Directory**: The lmxproxy repo is on windev at `C:\src\lmxproxy`
|
|
|
|
## Guardrails
|
|
|
|
1. **This is a v2 rebuild** — do not copy code from the v1 reference in `src-reference/`. Write fresh implementations guided by the design docs and the reference code's structure.
|
|
2. **Host targets .NET Framework 4.8, x86** — all code must use C# 9.0 language features maximum (`LangVersion` is `9.0` in the csproj). No file-scoped namespaces, no `required` keyword, no collection expressions in Host code.
|
|
3. **No new NuGet packages** — all required packages are already in the Host `.csproj` (`Microsoft.Extensions.Diagnostics.HealthChecks`, `Serilog`, `System.Threading.Channels`, `System.Text.Json` via framework).
|
|
4. **Namespace**: `ZB.MOM.WW.LmxProxy.Host` with sub-namespaces matching folder structure (e.g., `ZB.MOM.WW.LmxProxy.Host.Health`, `ZB.MOM.WW.LmxProxy.Host.Metrics`, `ZB.MOM.WW.LmxProxy.Host.Status`).
|
|
5. **All COM operations are on the STA thread** — health checks that read test tags must go through `MxAccessClient.ReadAsync()`, never directly touching COM objects.
|
|
6. **Build must pass after each step**: `dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86`
|
|
7. **Tests run on windev**: `dotnet test tests/ZB.MOM.WW.LmxProxy.Host.Tests --platform x86`
|
|
|
|
## Step 1: Create PerformanceMetrics
|
|
|
|
**File**: `src/ZB.MOM.WW.LmxProxy.Host/Metrics/PerformanceMetrics.cs`
|
|
|
|
Create the `PerformanceMetrics` class in namespace `ZB.MOM.WW.LmxProxy.Host.Metrics`.
|
|
|
|
### 1.1 OperationMetrics (nested or separate class in same file)
|
|
|
|
```csharp
|
|
namespace ZB.MOM.WW.LmxProxy.Host.Metrics
|
|
{
|
|
public class OperationMetrics
|
|
{
|
|
private readonly List<double> _durations = new List<double>();
|
|
private readonly object _lock = new object();
|
|
private long _totalCount;
|
|
private long _successCount;
|
|
private double _totalMilliseconds;
|
|
private double _minMilliseconds = double.MaxValue;
|
|
private double _maxMilliseconds;
|
|
|
|
public void Record(TimeSpan duration, bool success) { ... }
|
|
public MetricsStatistics GetStatistics() { ... }
|
|
}
|
|
}
|
|
```
|
|
|
|
Implementation details:
|
|
- `Record(TimeSpan duration, bool success)`: Inside `lock (_lock)`, increment `_totalCount`, conditionally increment `_successCount`, add `duration.TotalMilliseconds` to `_durations` list, update `_totalMilliseconds`, `_minMilliseconds`, `_maxMilliseconds`. If `_durations.Count > 1000`, call `_durations.RemoveAt(0)` to maintain rolling buffer.
|
|
- `GetStatistics()`: Inside `lock (_lock)`, return early with empty `MetricsStatistics` if `_totalCount == 0`. Otherwise sort `_durations`, compute p95 index as `(int)Math.Ceiling(sortedDurations.Count * 0.95) - 1`, clamp to `Math.Max(0, p95Index)`.
|
|
|
|
### 1.2 MetricsStatistics
|
|
|
|
```csharp
|
|
public class MetricsStatistics
|
|
{
|
|
public long TotalCount { get; set; }
|
|
public long SuccessCount { get; set; }
|
|
public double SuccessRate { get; set; }
|
|
public double AverageMilliseconds { get; set; }
|
|
public double MinMilliseconds { get; set; }
|
|
public double MaxMilliseconds { get; set; }
|
|
public double Percentile95Milliseconds { get; set; }
|
|
}
|
|
```
|
|
|
|
### 1.3 ITimingScope interface and TimingScope implementation
|
|
|
|
```csharp
|
|
public interface ITimingScope : IDisposable
|
|
{
|
|
void SetSuccess(bool success);
|
|
}
|
|
```
|
|
|
|
`TimingScope` is a private nested class inside `PerformanceMetrics`:
|
|
- Constructor takes `PerformanceMetrics metrics, string operationName`, starts a `Stopwatch`.
|
|
- `SetSuccess(bool success)` stores the flag (default `true`).
|
|
- `Dispose()`: stops stopwatch, calls `_metrics.RecordOperation(_operationName, _stopwatch.Elapsed, _success)`. Guard against double-dispose with `_disposed` flag.
|
|
|
|
### 1.4 PerformanceMetrics class
|
|
|
|
```csharp
|
|
public class PerformanceMetrics : IDisposable
|
|
{
|
|
private static readonly ILogger Logger = Log.ForContext<PerformanceMetrics>();
|
|
private readonly ConcurrentDictionary<string, OperationMetrics> _metrics = new ConcurrentDictionary<string, OperationMetrics>();
|
|
private readonly Timer _reportingTimer;
|
|
private bool _disposed;
|
|
|
|
public PerformanceMetrics()
|
|
{
|
|
_reportingTimer = new Timer(ReportMetrics, null, TimeSpan.FromSeconds(60), TimeSpan.FromSeconds(60));
|
|
}
|
|
|
|
public void RecordOperation(string operationName, TimeSpan duration, bool success = true) { ... }
|
|
public ITimingScope BeginOperation(string operationName) => new TimingScope(this, operationName);
|
|
public OperationMetrics? GetMetrics(string operationName) { ... }
|
|
public IReadOnlyDictionary<string, OperationMetrics> GetAllMetrics() { ... }
|
|
public Dictionary<string, MetricsStatistics> GetStatistics() { ... }
|
|
|
|
private void ReportMetrics(object? state) { ... } // Log each operation's stats at Information level
|
|
public void Dispose() { ... } // Dispose timer, call ReportMetrics one final time
|
|
}
|
|
```
|
|
|
|
`ReportMetrics` iterates `_metrics`, calls `GetStatistics()` on each, logs via Serilog structured logging with properties: `Operation`, `Count`, `SuccessRate`, `AverageMs`, `MinMs`, `MaxMs`, `P95Ms`.
|
|
|
|
### 1.5 Verify build
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86"
|
|
```
|
|
|
|
## Step 2: Create HealthCheckService
|
|
|
|
**File**: `src/ZB.MOM.WW.LmxProxy.Host/Health/HealthCheckService.cs`
|
|
|
|
Namespace: `ZB.MOM.WW.LmxProxy.Host.Health`
|
|
|
|
### 2.1 Basic HealthCheckService
|
|
|
|
```csharp
|
|
public class HealthCheckService : IHealthCheck
|
|
{
|
|
private static readonly ILogger Logger = Log.ForContext<HealthCheckService>();
|
|
private readonly IScadaClient _scadaClient;
|
|
private readonly SubscriptionManager _subscriptionManager;
|
|
private readonly PerformanceMetrics _performanceMetrics;
|
|
|
|
public HealthCheckService(
|
|
IScadaClient scadaClient,
|
|
SubscriptionManager subscriptionManager,
|
|
PerformanceMetrics performanceMetrics) { ... }
|
|
|
|
public Task<HealthCheckResult> CheckHealthAsync(
|
|
HealthCheckContext context,
|
|
CancellationToken cancellationToken = default) { ... }
|
|
}
|
|
```
|
|
|
|
Dependencies imported:
|
|
- `ZB.MOM.WW.LmxProxy.Host.Domain` for `IScadaClient`, `ConnectionState`
|
|
- `ZB.MOM.WW.LmxProxy.Host.Services` for `SubscriptionManager` (if still in that namespace after Phase 2/3; adjust import to match actual location)
|
|
- `ZB.MOM.WW.LmxProxy.Host.Metrics` for `PerformanceMetrics`
|
|
- `Microsoft.Extensions.Diagnostics.HealthChecks` for `IHealthCheck`, `HealthCheckResult`, `HealthCheckContext`
|
|
|
|
`CheckHealthAsync` logic:
|
|
1. Create `Dictionary<string, object> data`.
|
|
2. Read `_scadaClient.IsConnected` and `_scadaClient.ConnectionState` into `data["scada_connected"]` and `data["scada_connection_state"]`.
|
|
3. Get subscription stats via `_subscriptionManager.GetSubscriptionStats()` — store `TotalClients`, `TotalTags` in data.
|
|
4. Iterate `_performanceMetrics.GetAllMetrics()` to compute `totalOperations` and `averageSuccessRate`.
|
|
5. Store `total_operations` and `average_success_rate` in data.
|
|
6. Decision tree:
|
|
- If `!isConnected` → `HealthCheckResult.Unhealthy("SCADA client is not connected", data: data)`
|
|
- If `averageSuccessRate < 0.5 && totalOperations > 100` → `HealthCheckResult.Degraded(...)`
|
|
- If `subscriptionStats.TotalClients > 100` → `HealthCheckResult.Degraded(...)`
|
|
- Otherwise → `HealthCheckResult.Healthy("LmxProxy is healthy", data)`
|
|
7. Wrap everything in try/catch — on exception return `Unhealthy` with exception details.
|
|
|
|
### 2.2 DetailedHealthCheckService
|
|
|
|
In the same file or a separate file `src/ZB.MOM.WW.LmxProxy.Host/Health/DetailedHealthCheckService.cs`:
|
|
|
|
```csharp
|
|
public class DetailedHealthCheckService : IHealthCheck
|
|
{
|
|
private static readonly ILogger Logger = Log.ForContext<DetailedHealthCheckService>();
|
|
private readonly IScadaClient _scadaClient;
|
|
private readonly string _testTagAddress;
|
|
|
|
public DetailedHealthCheckService(IScadaClient scadaClient, string testTagAddress = "TestChildObject.TestBool") { ... }
|
|
|
|
public async Task<HealthCheckResult> CheckHealthAsync(
|
|
HealthCheckContext context,
|
|
CancellationToken cancellationToken = default) { ... }
|
|
}
|
|
```
|
|
|
|
`CheckHealthAsync` logic:
|
|
1. If `!_scadaClient.IsConnected` → return `Unhealthy`.
|
|
2. Try `Vtq vtq = await _scadaClient.ReadAsync(_testTagAddress, cancellationToken)`.
|
|
3. If `vtq.Quality != Quality.Good` → return `Degraded` with quality info.
|
|
4. If `DateTime.UtcNow - vtq.Timestamp > TimeSpan.FromMinutes(5)` → return `Degraded` (stale data).
|
|
5. Otherwise → `Healthy`.
|
|
6. Catch read exceptions → return `Degraded("Could not read test tag")`.
|
|
7. Catch all exceptions → return `Unhealthy`.
|
|
|
|
### 2.3 Verify build
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86"
|
|
```
|
|
|
|
## Step 3: Create StatusReportService
|
|
|
|
**File**: `src/ZB.MOM.WW.LmxProxy.Host/Status/StatusReportService.cs`
|
|
|
|
Namespace: `ZB.MOM.WW.LmxProxy.Host.Status`
|
|
|
|
### 3.1 Data model classes
|
|
|
|
Define in the same file (or a separate `StatusModels.cs` in the same folder):
|
|
|
|
```csharp
|
|
public class StatusData
|
|
{
|
|
public DateTime Timestamp { get; set; }
|
|
public string ServiceName { get; set; } = "";
|
|
public string Version { get; set; } = "";
|
|
public ConnectionStatus Connection { get; set; } = new ConnectionStatus();
|
|
public SubscriptionStatus Subscriptions { get; set; } = new SubscriptionStatus();
|
|
public PerformanceStatus Performance { get; set; } = new PerformanceStatus();
|
|
public HealthInfo Health { get; set; } = new HealthInfo();
|
|
public HealthInfo? DetailedHealth { get; set; }
|
|
}
|
|
|
|
public class ConnectionStatus
|
|
{
|
|
public bool IsConnected { get; set; }
|
|
public string State { get; set; } = "";
|
|
public string NodeName { get; set; } = "";
|
|
public string GalaxyName { get; set; } = "";
|
|
}
|
|
|
|
public class SubscriptionStatus
|
|
{
|
|
public int TotalClients { get; set; }
|
|
public int TotalTags { get; set; }
|
|
public int ActiveSubscriptions { get; set; }
|
|
}
|
|
|
|
public class PerformanceStatus
|
|
{
|
|
public long TotalOperations { get; set; }
|
|
public double AverageSuccessRate { get; set; }
|
|
public Dictionary<string, OperationStatus> Operations { get; set; } = new Dictionary<string, OperationStatus>();
|
|
}
|
|
|
|
public class OperationStatus
|
|
{
|
|
public long TotalCount { get; set; }
|
|
public double SuccessRate { get; set; }
|
|
public double AverageMilliseconds { get; set; }
|
|
public double MinMilliseconds { get; set; }
|
|
public double MaxMilliseconds { get; set; }
|
|
public double Percentile95Milliseconds { get; set; }
|
|
}
|
|
|
|
public class HealthInfo
|
|
{
|
|
public string Status { get; set; } = "";
|
|
public string Description { get; set; } = "";
|
|
public Dictionary<string, string> Data { get; set; } = new Dictionary<string, string>();
|
|
}
|
|
```
|
|
|
|
### 3.2 StatusReportService
|
|
|
|
```csharp
|
|
public class StatusReportService
|
|
{
|
|
private static readonly ILogger Logger = Log.ForContext<StatusReportService>();
|
|
private readonly IScadaClient _scadaClient;
|
|
private readonly SubscriptionManager _subscriptionManager;
|
|
private readonly PerformanceMetrics _performanceMetrics;
|
|
private readonly HealthCheckService _healthCheckService;
|
|
private readonly DetailedHealthCheckService? _detailedHealthCheckService;
|
|
|
|
public StatusReportService(
|
|
IScadaClient scadaClient,
|
|
SubscriptionManager subscriptionManager,
|
|
PerformanceMetrics performanceMetrics,
|
|
HealthCheckService healthCheckService,
|
|
DetailedHealthCheckService? detailedHealthCheckService = null) { ... }
|
|
|
|
public async Task<string> GenerateHtmlReportAsync() { ... }
|
|
public async Task<string> GenerateJsonReportAsync() { ... }
|
|
public async Task<bool> IsHealthyAsync() { ... }
|
|
private async Task<StatusData> CollectStatusDataAsync() { ... }
|
|
private static string GenerateHtmlFromStatusData(StatusData statusData) { ... }
|
|
private static string GenerateErrorHtml(Exception ex) { ... }
|
|
}
|
|
```
|
|
|
|
`CollectStatusDataAsync`:
|
|
- Populate `StatusData.Timestamp = DateTime.UtcNow`, `ServiceName = "ZB.MOM.WW.LmxProxy.Host"`, `Version` from `Assembly.GetExecutingAssembly().GetName().Version`.
|
|
- Connection info from `_scadaClient.IsConnected`, `_scadaClient.ConnectionState`.
|
|
- Subscription stats from `_subscriptionManager.GetSubscriptionStats()`.
|
|
- Performance stats from `_performanceMetrics.GetStatistics()` — include P95 in the `OperationStatus`.
|
|
- Health from `_healthCheckService.CheckHealthAsync(new HealthCheckContext())`.
|
|
- Detailed health from `_detailedHealthCheckService?.CheckHealthAsync(new HealthCheckContext())` if not null.
|
|
|
|
`GenerateJsonReportAsync`:
|
|
- Use `System.Text.Json.JsonSerializer.Serialize(statusData, new JsonSerializerOptions { WriteIndented = true, PropertyNamingPolicy = JsonNamingPolicy.CamelCase })`.
|
|
|
|
`GenerateHtmlFromStatusData`:
|
|
- Use `StringBuilder` to generate self-contained HTML.
|
|
- Include inline CSS (Bootstrap-like grid, status cards with color-coded left borders).
|
|
- Color coding: green (#28a745) for Healthy/Connected, yellow (#ffc107) for Degraded, red (#dc3545) for Unhealthy/Disconnected.
|
|
- Operations table with columns: Operation, Count, Success Rate, Avg (ms), Min (ms), Max (ms), P95 (ms).
|
|
- `<meta http-equiv="refresh" content="30">` for auto-refresh.
|
|
- Last updated timestamp at the bottom.
|
|
|
|
`IsHealthyAsync`:
|
|
- Run basic health check, return `result.Status == HealthStatus.Healthy`.
|
|
|
|
### 3.3 Verify build
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86"
|
|
```
|
|
|
|
## Step 4: Create StatusWebServer
|
|
|
|
**File**: `src/ZB.MOM.WW.LmxProxy.Host/Status/StatusWebServer.cs`
|
|
|
|
Namespace: `ZB.MOM.WW.LmxProxy.Host.Status`
|
|
|
|
```csharp
|
|
public class StatusWebServer : IDisposable
|
|
{
|
|
private static readonly ILogger Logger = Log.ForContext<StatusWebServer>();
|
|
private readonly WebServerConfiguration _configuration;
|
|
private readonly StatusReportService _statusReportService;
|
|
private HttpListener? _httpListener;
|
|
private CancellationTokenSource? _cancellationTokenSource;
|
|
private Task? _listenerTask;
|
|
private bool _disposed;
|
|
|
|
public StatusWebServer(WebServerConfiguration configuration, StatusReportService statusReportService) { ... }
|
|
|
|
public bool Start() { ... }
|
|
public bool Stop() { ... }
|
|
public void Dispose() { ... }
|
|
|
|
private async Task HandleRequestsAsync(CancellationToken cancellationToken) { ... }
|
|
private async Task HandleRequestAsync(HttpListenerContext context) { ... }
|
|
private async Task HandleStatusPageAsync(HttpListenerResponse response) { ... }
|
|
private async Task HandleStatusApiAsync(HttpListenerResponse response) { ... }
|
|
private async Task HandleHealthApiAsync(HttpListenerResponse response) { ... }
|
|
private static async Task WriteResponseAsync(HttpListenerResponse response, string content, string contentType) { ... }
|
|
}
|
|
```
|
|
|
|
### 4.1 Start()
|
|
|
|
1. If `!_configuration.Enabled`, log info and return `true`.
|
|
2. Create `HttpListener`, add prefix `_configuration.Prefix ?? $"http://+:{_configuration.Port}/"` (ensure trailing `/`).
|
|
3. Call `_httpListener.Start()`.
|
|
4. Create `_cancellationTokenSource = new CancellationTokenSource()`.
|
|
5. Start `_listenerTask = Task.Run(() => HandleRequestsAsync(_cancellationTokenSource.Token))`.
|
|
6. On exception, log error and return `false`.
|
|
|
|
### 4.2 Stop()
|
|
|
|
1. If not enabled or listener is null, return `true`.
|
|
2. Cancel `_cancellationTokenSource`.
|
|
3. Wait for `_listenerTask` with 5-second timeout.
|
|
4. Stop and close `_httpListener`.
|
|
|
|
### 4.3 HandleRequestsAsync
|
|
|
|
- Loop while not cancelled and listener is listening.
|
|
- `await _httpListener.GetContextAsync()` — on success, spawn `Task.Run` to handle.
|
|
- Catch `ObjectDisposedException` and `HttpListenerException(995)` as expected shutdown signals.
|
|
- On other errors, log and delay 1 second before continuing.
|
|
|
|
### 4.4 HandleRequestAsync routing
|
|
|
|
| Path (lowered) | Handler |
|
|
|---|---|
|
|
| `/` | `HandleStatusPageAsync` — calls `_statusReportService.GenerateHtmlReportAsync()`, content type `text/html; charset=utf-8` |
|
|
| `/api/status` | `HandleStatusApiAsync` — calls `_statusReportService.GenerateJsonReportAsync()`, content type `application/json; charset=utf-8` |
|
|
| `/api/health` | `HandleHealthApiAsync` — calls `_statusReportService.IsHealthyAsync()`, returns `"OK"` (200) or `"UNHEALTHY"` (503) as `text/plain` |
|
|
| Non-GET method | Return 405 Method Not Allowed |
|
|
| Unknown path | Return 404 Not Found |
|
|
| Exception | Return 500 Internal Server Error |
|
|
|
|
### 4.5 WriteResponseAsync
|
|
|
|
- Set `Content-Type`, add `Cache-Control: no-cache, no-store, must-revalidate`, `Pragma: no-cache`, `Expires: 0`.
|
|
- Convert content to UTF-8 bytes, set `ContentLength64`, write to `response.OutputStream`.
|
|
|
|
### 4.6 Dispose
|
|
|
|
- Guard with `_disposed` flag. Call `Stop()`. Dispose `_cancellationTokenSource` and close `_httpListener`.
|
|
|
|
### 4.7 Verify build
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86"
|
|
```
|
|
|
|
## Step 5: Wire into LmxProxyService
|
|
|
|
**File**: `src/ZB.MOM.WW.LmxProxy.Host/LmxProxyService.cs`
|
|
|
|
This file already exists. Modify the `Start()` method to create and wire the new components. The v2 rebuild should create these fresh, but the wiring pattern follows the same order as the reference.
|
|
|
|
### 5.1 Add using directives
|
|
|
|
```csharp
|
|
using ZB.MOM.WW.LmxProxy.Host.Health;
|
|
using ZB.MOM.WW.LmxProxy.Host.Metrics;
|
|
using ZB.MOM.WW.LmxProxy.Host.Status;
|
|
```
|
|
|
|
### 5.2 Add fields
|
|
|
|
```csharp
|
|
private PerformanceMetrics? _performanceMetrics;
|
|
private HealthCheckService? _healthCheckService;
|
|
private DetailedHealthCheckService? _detailedHealthCheckService;
|
|
private StatusReportService? _statusReportService;
|
|
private StatusWebServer? _statusWebServer;
|
|
```
|
|
|
|
### 5.3 In Start(), after SessionManager and SubscriptionManager creation
|
|
|
|
```csharp
|
|
// Create performance metrics
|
|
_performanceMetrics = new PerformanceMetrics();
|
|
|
|
// Create health check services
|
|
_healthCheckService = new HealthCheckService(_scadaClient, _subscriptionManager, _performanceMetrics);
|
|
_detailedHealthCheckService = new DetailedHealthCheckService(_scadaClient);
|
|
|
|
// Create status report service
|
|
_statusReportService = new StatusReportService(
|
|
_scadaClient, _subscriptionManager, _performanceMetrics,
|
|
_healthCheckService, _detailedHealthCheckService);
|
|
|
|
// Start status web server
|
|
_statusWebServer = new StatusWebServer(_configuration.WebServer, _statusReportService);
|
|
if (!_statusWebServer.Start())
|
|
{
|
|
Logger.Warning("Status web server failed to start — continuing without it");
|
|
}
|
|
```
|
|
|
|
### 5.4 In Stop(), before gRPC server shutdown
|
|
|
|
```csharp
|
|
// Stop status web server
|
|
_statusWebServer?.Stop();
|
|
|
|
// Dispose performance metrics
|
|
_performanceMetrics?.Dispose();
|
|
```
|
|
|
|
### 5.5 Pass _performanceMetrics to ScadaGrpcService constructor
|
|
|
|
Ensure `ScadaGrpcService` receives `_performanceMetrics` so it can record timings on each RPC call. The gRPC service should call `_performanceMetrics.BeginOperation("Read")` (etc.) and dispose the timing scope at the end of each RPC handler.
|
|
|
|
### 5.6 Verify build
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build src/ZB.MOM.WW.LmxProxy.Host --platform x86"
|
|
```
|
|
|
|
## Step 6: Unit Tests
|
|
|
|
**Project**: `tests/ZB.MOM.WW.LmxProxy.Host.Tests/`
|
|
|
|
If this project does not exist yet, create it:
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet new xunit -n ZB.MOM.WW.LmxProxy.Host.Tests -o tests/ZB.MOM.WW.LmxProxy.Host.Tests --framework net48"
|
|
```
|
|
|
|
**Csproj adjustments** for `tests/ZB.MOM.WW.LmxProxy.Host.Tests/ZB.MOM.WW.LmxProxy.Host.Tests.csproj`:
|
|
- `<TargetFramework>net48</TargetFramework>`
|
|
- `<PlatformTarget>x86</PlatformTarget>`
|
|
- `<LangVersion>9.0</LangVersion>`
|
|
- Add `<ProjectReference Include="..\..\src\ZB.MOM.WW.LmxProxy.Host\ZB.MOM.WW.LmxProxy.Host.csproj" />`
|
|
- Add `<PackageReference Include="xunit" Version="2.9.3" />`
|
|
- Add `<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />`
|
|
- Add `<PackageReference Include="NSubstitute" Version="5.3.0" />` (for mocking IScadaClient)
|
|
- Add `<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />`
|
|
|
|
**Also add to solution** in `ZB.MOM.WW.LmxProxy.slnx`:
|
|
```xml
|
|
<Folder Name="/tests/">
|
|
<Project Path="tests/ZB.MOM.WW.LmxProxy.Host.Tests/ZB.MOM.WW.LmxProxy.Host.Tests.csproj" />
|
|
</Folder>
|
|
```
|
|
|
|
### 6.1 PerformanceMetrics Tests
|
|
|
|
**File**: `tests/ZB.MOM.WW.LmxProxy.Host.Tests/Metrics/PerformanceMetricsTests.cs`
|
|
|
|
```csharp
|
|
namespace ZB.MOM.WW.LmxProxy.Host.Tests.Metrics
|
|
{
|
|
public class PerformanceMetricsTests
|
|
{
|
|
[Fact]
|
|
public void RecordOperation_TracksCountAndDuration()
|
|
// Record 5 operations, verify GetStatistics returns TotalCount=5
|
|
|
|
[Fact]
|
|
public void RecordOperation_TracksSuccessAndFailure()
|
|
// Record 3 success + 2 failure, verify SuccessRate == 0.6
|
|
|
|
[Fact]
|
|
public void GetStatistics_CalculatesP95Correctly()
|
|
// Record 100 operations with known durations (1ms through 100ms)
|
|
// Verify P95 is approximately 95ms
|
|
|
|
[Fact]
|
|
public void RollingBuffer_CapsAt1000Samples()
|
|
// Record 1500 operations, verify _durations list doesn't exceed 1000
|
|
// (test via GetStatistics behavior — TotalCount is 1500 but percentile computed from 1000)
|
|
|
|
[Fact]
|
|
public void BeginOperation_RecordsDurationOnDispose()
|
|
// Use BeginOperation, await Task.Delay(50), dispose scope
|
|
// Verify recorded duration >= 50ms
|
|
|
|
[Fact]
|
|
public void TimingScope_DefaultsToSuccess()
|
|
// BeginOperation + dispose without calling SetSuccess
|
|
// Verify SuccessCount == 1
|
|
|
|
[Fact]
|
|
public void TimingScope_RespectsSetSuccessFalse()
|
|
// BeginOperation, SetSuccess(false), dispose
|
|
// Verify SuccessCount == 0, TotalCount == 1
|
|
|
|
[Fact]
|
|
public void GetMetrics_ReturnsNullForUnknownOperation()
|
|
|
|
[Fact]
|
|
public void GetAllMetrics_ReturnsAllTrackedOperations()
|
|
}
|
|
}
|
|
```
|
|
|
|
### 6.2 HealthCheckService Tests
|
|
|
|
**File**: `tests/ZB.MOM.WW.LmxProxy.Host.Tests/Health/HealthCheckServiceTests.cs`
|
|
|
|
Use NSubstitute to mock `IScadaClient`. Create a real `PerformanceMetrics` instance and a real or mock `SubscriptionManager` (depends on Phase 2/3 implementation — if `SubscriptionManager` has an interface, mock it; if not, use the `GetSubscriptionStats()` approach with a concrete instance).
|
|
|
|
```csharp
|
|
namespace ZB.MOM.WW.LmxProxy.Host.Tests.Health
|
|
{
|
|
public class HealthCheckServiceTests
|
|
{
|
|
[Fact]
|
|
public async Task ReturnsHealthy_WhenConnectedAndNormalMetrics()
|
|
// Mock: IsConnected=true, ConnectionState=Connected
|
|
// SubscriptionStats: TotalClients=5, TotalTags=10
|
|
// PerformanceMetrics: record some successes
|
|
// Assert: HealthStatus.Healthy
|
|
|
|
[Fact]
|
|
public async Task ReturnsUnhealthy_WhenNotConnected()
|
|
// Mock: IsConnected=false
|
|
// Assert: HealthStatus.Unhealthy, description contains "not connected"
|
|
|
|
[Fact]
|
|
public async Task ReturnsDegraded_WhenSuccessRateBelow50Percent()
|
|
// Mock: IsConnected=true
|
|
// Record 200 operations with 40% success rate
|
|
// Assert: HealthStatus.Degraded
|
|
|
|
[Fact]
|
|
public async Task ReturnsDegraded_WhenClientCountOver100()
|
|
// Mock: IsConnected=true, SubscriptionStats.TotalClients=150
|
|
// Assert: HealthStatus.Degraded
|
|
|
|
[Fact]
|
|
public async Task DoesNotFlagLowSuccessRate_Under100Operations()
|
|
// Record 50 operations with 0% success rate
|
|
// Assert: still Healthy (threshold is > 100 total ops)
|
|
}
|
|
|
|
public class DetailedHealthCheckServiceTests
|
|
{
|
|
[Fact]
|
|
public async Task ReturnsUnhealthy_WhenNotConnected()
|
|
|
|
[Fact]
|
|
public async Task ReturnsHealthy_WhenTestTagGoodAndRecent()
|
|
// Mock ReadAsync returns Good quality with recent timestamp
|
|
// Assert: Healthy
|
|
|
|
[Fact]
|
|
public async Task ReturnsDegraded_WhenTestTagQualityNotGood()
|
|
// Mock ReadAsync returns Uncertain quality
|
|
// Assert: Degraded
|
|
|
|
[Fact]
|
|
public async Task ReturnsDegraded_WhenTestTagTimestampStale()
|
|
// Mock ReadAsync returns Good quality but timestamp 10 minutes ago
|
|
// Assert: Degraded
|
|
|
|
[Fact]
|
|
public async Task ReturnsDegraded_WhenTestTagReadThrows()
|
|
// Mock ReadAsync throws exception
|
|
// Assert: Degraded
|
|
}
|
|
}
|
|
```
|
|
|
|
### 6.3 StatusReportService Tests
|
|
|
|
**File**: `tests/ZB.MOM.WW.LmxProxy.Host.Tests/Status/StatusReportServiceTests.cs`
|
|
|
|
```csharp
|
|
namespace ZB.MOM.WW.LmxProxy.Host.Tests.Status
|
|
{
|
|
public class StatusReportServiceTests
|
|
{
|
|
[Fact]
|
|
public async Task GenerateJsonReportAsync_ReturnsCamelCaseJson()
|
|
// Verify JSON contains "serviceName", "connection", "isConnected" (camelCase)
|
|
|
|
[Fact]
|
|
public async Task GenerateHtmlReportAsync_ContainsAutoRefresh()
|
|
// Verify HTML contains <meta http-equiv="refresh" content="30">
|
|
|
|
[Fact]
|
|
public async Task IsHealthyAsync_ReturnsTrueWhenHealthy()
|
|
|
|
[Fact]
|
|
public async Task IsHealthyAsync_ReturnsFalseWhenUnhealthy()
|
|
|
|
[Fact]
|
|
public async Task GenerateJsonReportAsync_IncludesPerformanceMetrics()
|
|
// Record some operations, verify JSON includes operation names and stats
|
|
}
|
|
}
|
|
```
|
|
|
|
### 6.4 Run tests
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet test tests/ZB.MOM.WW.LmxProxy.Host.Tests --platform x86 --verbosity normal"
|
|
```
|
|
|
|
## Step 7: Build Verification
|
|
|
|
Run full solution build and tests:
|
|
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build ZB.MOM.WW.LmxProxy.slnx && dotnet test --verbosity normal"
|
|
```
|
|
|
|
If the test project is .NET 4.8 x86, you may need:
|
|
```bash
|
|
ssh windev "cd C:\src\lmxproxy && dotnet build ZB.MOM.WW.LmxProxy.slnx --platform x86 && dotnet test tests/ZB.MOM.WW.LmxProxy.Host.Tests --platform x86"
|
|
```
|
|
|
|
## Completion Criteria
|
|
|
|
- [ ] `PerformanceMetrics` class with `OperationMetrics`, `MetricsStatistics`, `ITimingScope` in `src/ZB.MOM.WW.LmxProxy.Host/Metrics/`
|
|
- [ ] `HealthCheckService` and `DetailedHealthCheckService` in `src/ZB.MOM.WW.LmxProxy.Host/Health/`
|
|
- [ ] `StatusReportService` with data model classes in `src/ZB.MOM.WW.LmxProxy.Host/Status/`
|
|
- [ ] `StatusWebServer` with HTML dashboard, JSON status, and health endpoints in `src/ZB.MOM.WW.LmxProxy.Host/Status/`
|
|
- [ ] All components wired into `LmxProxyService.Start()` / `Stop()`
|
|
- [ ] `ScadaGrpcService` uses `PerformanceMetrics.BeginOperation()` for Read, ReadBatch, Write, WriteBatch RPCs
|
|
- [ ] Unit tests for PerformanceMetrics (recording, percentile, rolling buffer, timing scope)
|
|
- [ ] Unit tests for HealthCheckService (healthy, unhealthy, degraded transitions)
|
|
- [ ] Unit tests for DetailedHealthCheckService (connected, quality, staleness)
|
|
- [ ] Unit tests for StatusReportService (JSON format, HTML format, health aggregation)
|
|
- [ ] Solution builds without errors: `dotnet build ZB.MOM.WW.LmxProxy.slnx`
|
|
- [ ] All tests pass: `dotnet test`
|