Phase 6.4 Stream A + B data layer — UnsImpactAnalyzer + EquipmentCsvImporter (parser)

Ships the pure-logic data layer of Phase 6.4. Blazor UI pieces
(UnsTab drag/drop, CSV import modal, preview table, FinaliseImportBatch txn,
staging tables) are deferred to visual-compliance follow-ups (tasks #153,
#155, #157).

Admin.Services additions:

- UnsImpactAnalyzer.Analyze(snapshot, move) — pure-function, no I/O. Three
  move variants: LineMove, AreaRename, LineMerge. Returns UnsImpactPreview
  with AffectedEquipmentCount + AffectedTagCount + CascadeWarnings +
  RevisionToken + HumanReadableSummary the Admin UI shows in the confirm
  modal. Cross-cluster moves rejected with CrossClusterMoveRejectedException
  per decision #82. Missing source/target throws UnsMoveValidationException.
  Surfaces sibling-line same-name ambiguity as a cascade warning.
- DraftRevisionToken — opaque revision fingerprint. Preview captures the
  token; Confirm compares it. The 409-concurrent-edit UX plumbs through on
  the Razor-page follow-up (task #153). Matches(other) is null-safe.
- UnsTreeSnapshot + UnsAreaSummary + UnsLineSummary — snapshot shape the
  caller hands to the analyzer. Tests build them in-memory without a DB.

- EquipmentCsvImporter.Parse(csvText) — RFC 4180 CSV parser per decision #95.
  Version-marker contract: line 1 must be "# OtOpcUaCsv v1" (future shapes
  bump the version). Required columns from decision #117 + optional columns
  from decision #139. Rejects unknown columns, duplicate column names,
  blank required fields, duplicate ZTags within the file. Quoted-field
  handling supports embedded commas + escaped "" quotes. Returns
  EquipmentCsvParseResult { AcceptedRows, RejectedRows } so the preview
  modal renders accept/reject counts without re-parsing.

Tests (22 new, all pass):

- UnsImpactAnalyzerTests (9): line move counts equipment + tags; cross-
  cluster throws; unknown source/target throws validation; ambiguous same-
  name target raises warning; area rename sums across lines; line merge
  cross-area warns; same-area merge no warning; DraftRevisionToken matches
  semantics.
- EquipmentCsvImporterTests (13): empty file throws; missing version marker;
  missing required column; unknown column; duplicate column; valid single
  row round-trips; optional columns populate when present; blank required
  field rejects row; duplicate ZTag rejects second; RFC 4180 quoted fields
  with commas + escaped quotes; mismatched column count rejects; blank
  lines between rows ignored; required + optional column constants match
  decisions #117 + #139 exactly.

Full solution dotnet test: 1159 passing (Phase 6.3 = 1137, Phase 6.4 A+B
data = +22). Pre-existing Client.CLI Subscribe flake unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-04-19 10:09:47 -04:00
parent 4901b78e9a
commit 560a961cca
4 changed files with 814 additions and 0 deletions

View File

@@ -0,0 +1,259 @@
using System.Globalization;
using System.Text;
namespace ZB.MOM.WW.OtOpcUa.Admin.Services;
/// <summary>
/// RFC 4180 CSV parser for equipment import per decision #95 and Phase 6.4 Stream B.1.
/// Produces a validated <see cref="EquipmentCsvParseResult"/> the caller (CSV import
/// modal + staging tables) consumes. Pure-parser concern — no DB access, no staging
/// writes; those live in the follow-up Stream B.2 work.
/// </summary>
/// <remarks>
/// <para><b>Header contract</b>: line 1 must be exactly <c># OtOpcUaCsv v1</c> (version
/// marker). Line 2 is the column header row. Unknown columns are rejected; required
/// columns must all be present. The version bump handshake lets future shapes parse
/// without ambiguity — v2 files go through a different parser variant.</para>
///
/// <para><b>Required columns</b> per decision #117: ZTag, MachineCode, SAPID,
/// EquipmentId, EquipmentUuid, Name, UnsAreaName, UnsLineName.</para>
///
/// <para><b>Optional columns</b> per decision #139: Manufacturer, Model, SerialNumber,
/// HardwareRevision, SoftwareRevision, YearOfConstruction, AssetLocation,
/// ManufacturerUri, DeviceManualUri.</para>
///
/// <para><b>Row validation</b>: blank required field → rejected; duplicate ZTag within
/// the same file → rejected. Duplicate against the DB isn't detected here — the
/// staged-import finalize step (Stream B.4) catches that.</para>
/// </remarks>
public static class EquipmentCsvImporter
{
public const string VersionMarker = "# OtOpcUaCsv v1";
public static IReadOnlyList<string> RequiredColumns { get; } = new[]
{
"ZTag", "MachineCode", "SAPID", "EquipmentId", "EquipmentUuid",
"Name", "UnsAreaName", "UnsLineName",
};
public static IReadOnlyList<string> OptionalColumns { get; } = new[]
{
"Manufacturer", "Model", "SerialNumber", "HardwareRevision", "SoftwareRevision",
"YearOfConstruction", "AssetLocation", "ManufacturerUri", "DeviceManualUri",
};
public static EquipmentCsvParseResult Parse(string csvText)
{
ArgumentNullException.ThrowIfNull(csvText);
var rows = SplitLines(csvText);
if (rows.Count == 0)
throw new InvalidCsvFormatException("CSV is empty.");
if (!string.Equals(rows[0].Trim(), VersionMarker, StringComparison.Ordinal))
throw new InvalidCsvFormatException(
$"CSV header line 1 must be exactly '{VersionMarker}' — got '{rows[0]}'. " +
"Files without the version marker are rejected so future-format files don't parse ambiguously.");
if (rows.Count < 2)
throw new InvalidCsvFormatException("CSV has no column header row (line 2) or data rows.");
var headerCells = SplitCsvRow(rows[1]);
ValidateHeader(headerCells);
var accepted = new List<EquipmentCsvRow>();
var rejected = new List<EquipmentCsvRowError>();
var ztagsSeen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var colIndex = headerCells
.Select((name, idx) => (name, idx))
.ToDictionary(t => t.name, t => t.idx, StringComparer.OrdinalIgnoreCase);
for (var i = 2; i < rows.Count; i++)
{
if (string.IsNullOrWhiteSpace(rows[i])) continue;
try
{
var cells = SplitCsvRow(rows[i]);
if (cells.Length != headerCells.Length)
{
rejected.Add(new EquipmentCsvRowError(
LineNumber: i + 1,
Reason: $"Column count {cells.Length} != header count {headerCells.Length}."));
continue;
}
var row = BuildRow(cells, colIndex);
var missing = RequiredColumns.Where(c => string.IsNullOrWhiteSpace(GetCell(row, c))).ToList();
if (missing.Count > 0)
{
rejected.Add(new EquipmentCsvRowError(i + 1, $"Blank required column(s): {string.Join(", ", missing)}"));
continue;
}
if (!ztagsSeen.Add(row.ZTag))
{
rejected.Add(new EquipmentCsvRowError(i + 1, $"Duplicate ZTag '{row.ZTag}' within file."));
continue;
}
accepted.Add(row);
}
catch (InvalidCsvFormatException ex)
{
rejected.Add(new EquipmentCsvRowError(i + 1, ex.Message));
}
}
return new EquipmentCsvParseResult(accepted, rejected);
}
private static void ValidateHeader(string[] headerCells)
{
var seen = new HashSet<string>(headerCells, StringComparer.OrdinalIgnoreCase);
// Missing required
var missingRequired = RequiredColumns.Where(r => !seen.Contains(r)).ToList();
if (missingRequired.Count > 0)
throw new InvalidCsvFormatException($"Header is missing required column(s): {string.Join(", ", missingRequired)}");
// Unknown columns (not in required optional)
var known = new HashSet<string>(RequiredColumns.Concat(OptionalColumns), StringComparer.OrdinalIgnoreCase);
var unknown = headerCells.Where(c => !known.Contains(c)).ToList();
if (unknown.Count > 0)
throw new InvalidCsvFormatException(
$"Header has unknown column(s): {string.Join(", ", unknown)}. " +
"Bump the version marker to define a new shape before adding columns.");
// Duplicates
var dupe = headerCells.GroupBy(c => c, StringComparer.OrdinalIgnoreCase).FirstOrDefault(g => g.Count() > 1);
if (dupe is not null)
throw new InvalidCsvFormatException($"Header has duplicate column '{dupe.Key}'.");
}
private static EquipmentCsvRow BuildRow(string[] cells, Dictionary<string, int> colIndex) => new()
{
ZTag = cells[colIndex["ZTag"]],
MachineCode = cells[colIndex["MachineCode"]],
SAPID = cells[colIndex["SAPID"]],
EquipmentId = cells[colIndex["EquipmentId"]],
EquipmentUuid = cells[colIndex["EquipmentUuid"]],
Name = cells[colIndex["Name"]],
UnsAreaName = cells[colIndex["UnsAreaName"]],
UnsLineName = cells[colIndex["UnsLineName"]],
Manufacturer = colIndex.TryGetValue("Manufacturer", out var mi) ? cells[mi] : null,
Model = colIndex.TryGetValue("Model", out var moi) ? cells[moi] : null,
SerialNumber = colIndex.TryGetValue("SerialNumber", out var si) ? cells[si] : null,
HardwareRevision = colIndex.TryGetValue("HardwareRevision", out var hi) ? cells[hi] : null,
SoftwareRevision = colIndex.TryGetValue("SoftwareRevision", out var swi) ? cells[swi] : null,
YearOfConstruction = colIndex.TryGetValue("YearOfConstruction", out var yi) ? cells[yi] : null,
AssetLocation = colIndex.TryGetValue("AssetLocation", out var ai) ? cells[ai] : null,
ManufacturerUri = colIndex.TryGetValue("ManufacturerUri", out var mui) ? cells[mui] : null,
DeviceManualUri = colIndex.TryGetValue("DeviceManualUri", out var dui) ? cells[dui] : null,
};
private static string GetCell(EquipmentCsvRow row, string colName) => colName switch
{
"ZTag" => row.ZTag,
"MachineCode" => row.MachineCode,
"SAPID" => row.SAPID,
"EquipmentId" => row.EquipmentId,
"EquipmentUuid" => row.EquipmentUuid,
"Name" => row.Name,
"UnsAreaName" => row.UnsAreaName,
"UnsLineName" => row.UnsLineName,
_ => string.Empty,
};
/// <summary>Split the raw text on line boundaries. Handles \r\n + \n + \r.</summary>
private static List<string> SplitLines(string csv) =>
csv.Split(["\r\n", "\n", "\r"], StringSplitOptions.None).ToList();
/// <summary>Split one CSV row with RFC 4180 quoted-field handling.</summary>
private static string[] SplitCsvRow(string row)
{
var cells = new List<string>();
var sb = new StringBuilder();
var inQuotes = false;
for (var i = 0; i < row.Length; i++)
{
var ch = row[i];
if (inQuotes)
{
if (ch == '"')
{
// Escaped quote "" inside quoted field.
if (i + 1 < row.Length && row[i + 1] == '"')
{
sb.Append('"');
i++;
}
else
{
inQuotes = false;
}
}
else
{
sb.Append(ch);
}
}
else
{
if (ch == ',')
{
cells.Add(sb.ToString());
sb.Clear();
}
else if (ch == '"' && sb.Length == 0)
{
inQuotes = true;
}
else
{
sb.Append(ch);
}
}
}
cells.Add(sb.ToString());
return cells.ToArray();
}
}
/// <summary>One parsed equipment row with required + optional fields.</summary>
public sealed class EquipmentCsvRow
{
// Required (decision #117)
public required string ZTag { get; init; }
public required string MachineCode { get; init; }
public required string SAPID { get; init; }
public required string EquipmentId { get; init; }
public required string EquipmentUuid { get; init; }
public required string Name { get; init; }
public required string UnsAreaName { get; init; }
public required string UnsLineName { get; init; }
// Optional (decision #139 — OPC 40010 Identification fields)
public string? Manufacturer { get; init; }
public string? Model { get; init; }
public string? SerialNumber { get; init; }
public string? HardwareRevision { get; init; }
public string? SoftwareRevision { get; init; }
public string? YearOfConstruction { get; init; }
public string? AssetLocation { get; init; }
public string? ManufacturerUri { get; init; }
public string? DeviceManualUri { get; init; }
}
/// <summary>One row-level rejection captured by the parser. Line-number is 1-based in the source file.</summary>
public sealed record EquipmentCsvRowError(int LineNumber, string Reason);
/// <summary>Parser output — accepted rows land in staging; rejected rows surface in the preview modal.</summary>
public sealed record EquipmentCsvParseResult(
IReadOnlyList<EquipmentCsvRow> AcceptedRows,
IReadOnlyList<EquipmentCsvRowError> RejectedRows);
/// <summary>Thrown for file-level format problems (missing version marker, bad header, etc.).</summary>
public sealed class InvalidCsvFormatException(string message) : Exception(message);

View File

@@ -0,0 +1,213 @@
namespace ZB.MOM.WW.OtOpcUa.Admin.Services;
/// <summary>
/// Pure-function impact preview for UNS structural moves per Phase 6.4 Stream A.2. Given
/// a <see cref="UnsMoveOperation"/> plus a snapshot of the draft's UNS tree and its
/// equipment + tag counts, returns an <see cref="UnsImpactPreview"/> the Admin UI shows
/// in a confirmation modal before committing the move.
/// </summary>
/// <remarks>
/// <para>Stateless + deterministic — testable without EF or a live draft. The caller
/// (Razor page) loads the draft's snapshot via the normal Configuration services, passes
/// it in, and the analyzer counts + categorises the impact. The returned
/// <see cref="UnsImpactPreview.RevisionToken"/> is the token the caller must re-check at
/// confirm time; a mismatch means another operator mutated the draft between preview +
/// confirm and the operation needs to be refreshed (decision on concurrent-edit safety
/// in Phase 6.4 Scope).</para>
///
/// <para>Cross-cluster moves are rejected here (decision #82) — equipment is
/// cluster-scoped; the UI disables the drop target and surfaces an Export/Import workflow
/// toast instead.</para>
/// </remarks>
public static class UnsImpactAnalyzer
{
/// <summary>Run the analyzer. Returns a populated preview or throws for invalid operations.</summary>
public static UnsImpactPreview Analyze(UnsTreeSnapshot snapshot, UnsMoveOperation move)
{
ArgumentNullException.ThrowIfNull(snapshot);
ArgumentNullException.ThrowIfNull(move);
// Cross-cluster guard — the analyzer refuses rather than silently re-homing.
if (!string.Equals(move.SourceClusterId, move.TargetClusterId, StringComparison.OrdinalIgnoreCase))
throw new CrossClusterMoveRejectedException(
"Equipment is cluster-scoped (decision #82). Use Export → Import to migrate equipment " +
"across clusters; drag/drop rejected.");
return move.Kind switch
{
UnsMoveKind.LineMove => AnalyzeLineMove(snapshot, move),
UnsMoveKind.AreaRename => AnalyzeAreaRename(snapshot, move),
UnsMoveKind.LineMerge => AnalyzeLineMerge(snapshot, move),
_ => throw new ArgumentOutOfRangeException(nameof(move), move.Kind, $"Unsupported move kind {move.Kind}"),
};
}
private static UnsImpactPreview AnalyzeLineMove(UnsTreeSnapshot snapshot, UnsMoveOperation move)
{
var line = snapshot.FindLine(move.SourceLineId!)
?? throw new UnsMoveValidationException($"Source line '{move.SourceLineId}' not found in draft {snapshot.DraftGenerationId}.");
var targetArea = snapshot.FindArea(move.TargetAreaId!)
?? throw new UnsMoveValidationException($"Target area '{move.TargetAreaId}' not found in draft {snapshot.DraftGenerationId}.");
var warnings = new List<string>();
if (targetArea.LineIds.Contains(line.LineId, StringComparer.OrdinalIgnoreCase))
warnings.Add($"Target area '{targetArea.Name}' already contains line '{line.Name}' — dropping a no-op move.");
// If the target area has a line with the same display name as the mover, warn about
// visual ambiguity even though the IDs differ (operators frequently reuse line names).
if (targetArea.LineIds.Any(lid =>
snapshot.FindLine(lid) is { } sibling &&
string.Equals(sibling.Name, line.Name, StringComparison.OrdinalIgnoreCase) &&
!string.Equals(sibling.LineId, line.LineId, StringComparison.OrdinalIgnoreCase)))
{
warnings.Add($"Target area '{targetArea.Name}' already has a line named '{line.Name}'. Consider renaming before the move.");
}
return new UnsImpactPreview
{
AffectedEquipmentCount = line.EquipmentCount,
AffectedTagCount = line.TagCount,
CascadeWarnings = warnings,
RevisionToken = snapshot.RevisionToken,
HumanReadableSummary =
$"Moving line '{line.Name}' from area '{snapshot.FindAreaByLineId(line.LineId)?.Name ?? "?"}' " +
$"to '{targetArea.Name}' will re-home {line.EquipmentCount} equipment + re-parent {line.TagCount} tags.",
};
}
private static UnsImpactPreview AnalyzeAreaRename(UnsTreeSnapshot snapshot, UnsMoveOperation move)
{
var area = snapshot.FindArea(move.SourceAreaId!)
?? throw new UnsMoveValidationException($"Source area '{move.SourceAreaId}' not found in draft {snapshot.DraftGenerationId}.");
var affectedEquipment = area.LineIds
.Select(lid => snapshot.FindLine(lid)?.EquipmentCount ?? 0)
.Sum();
var affectedTags = area.LineIds
.Select(lid => snapshot.FindLine(lid)?.TagCount ?? 0)
.Sum();
return new UnsImpactPreview
{
AffectedEquipmentCount = affectedEquipment,
AffectedTagCount = affectedTags,
CascadeWarnings = [],
RevisionToken = snapshot.RevisionToken,
HumanReadableSummary =
$"Renaming area '{area.Name}' → '{move.NewName}' cascades to {area.LineIds.Count} lines / " +
$"{affectedEquipment} equipment / {affectedTags} tags.",
};
}
private static UnsImpactPreview AnalyzeLineMerge(UnsTreeSnapshot snapshot, UnsMoveOperation move)
{
var src = snapshot.FindLine(move.SourceLineId!)
?? throw new UnsMoveValidationException($"Source line '{move.SourceLineId}' not found.");
var dst = snapshot.FindLine(move.TargetLineId!)
?? throw new UnsMoveValidationException($"Target line '{move.TargetLineId}' not found.");
var warnings = new List<string>();
if (!string.Equals(snapshot.FindAreaByLineId(src.LineId)?.AreaId,
snapshot.FindAreaByLineId(dst.LineId)?.AreaId,
StringComparison.OrdinalIgnoreCase))
{
warnings.Add($"Lines '{src.Name}' and '{dst.Name}' are in different areas. The merge will re-parent equipment + tags into '{dst.Name}'s area.");
}
return new UnsImpactPreview
{
AffectedEquipmentCount = src.EquipmentCount,
AffectedTagCount = src.TagCount,
CascadeWarnings = warnings,
RevisionToken = snapshot.RevisionToken,
HumanReadableSummary =
$"Merging line '{src.Name}' into '{dst.Name}': {src.EquipmentCount} equipment + {src.TagCount} tags re-parent. " +
$"The source line is deleted at commit.",
};
}
}
/// <summary>Kind of UNS structural move the analyzer understands.</summary>
public enum UnsMoveKind
{
/// <summary>Drag a whole line from one area to another.</summary>
LineMove,
/// <summary>Rename an area (cascades to the UNS paths of every equipment + tag below it).</summary>
AreaRename,
/// <summary>Merge two lines into one; source line's equipment + tags are re-parented.</summary>
LineMerge,
}
/// <summary>One UNS structural move request.</summary>
/// <param name="Kind">Move variant — selects which source + target fields are required.</param>
/// <param name="SourceClusterId">Cluster of the source node. Must match <see cref="TargetClusterId"/> (decision #82).</param>
/// <param name="TargetClusterId">Cluster of the target node.</param>
/// <param name="SourceAreaId">Source area id for <see cref="UnsMoveKind.AreaRename"/>.</param>
/// <param name="SourceLineId">Source line id for <see cref="UnsMoveKind.LineMove"/> / <see cref="UnsMoveKind.LineMerge"/>.</param>
/// <param name="TargetAreaId">Target area id for <see cref="UnsMoveKind.LineMove"/>.</param>
/// <param name="TargetLineId">Target line id for <see cref="UnsMoveKind.LineMerge"/>.</param>
/// <param name="NewName">New display name for <see cref="UnsMoveKind.AreaRename"/>.</param>
public sealed record UnsMoveOperation(
UnsMoveKind Kind,
string SourceClusterId,
string TargetClusterId,
string? SourceAreaId = null,
string? SourceLineId = null,
string? TargetAreaId = null,
string? TargetLineId = null,
string? NewName = null);
/// <summary>Snapshot of the UNS tree + counts the analyzer walks.</summary>
public sealed class UnsTreeSnapshot
{
public required long DraftGenerationId { get; init; }
public required DraftRevisionToken RevisionToken { get; init; }
public required IReadOnlyList<UnsAreaSummary> Areas { get; init; }
public required IReadOnlyList<UnsLineSummary> Lines { get; init; }
public UnsAreaSummary? FindArea(string areaId) =>
Areas.FirstOrDefault(a => string.Equals(a.AreaId, areaId, StringComparison.OrdinalIgnoreCase));
public UnsLineSummary? FindLine(string lineId) =>
Lines.FirstOrDefault(l => string.Equals(l.LineId, lineId, StringComparison.OrdinalIgnoreCase));
public UnsAreaSummary? FindAreaByLineId(string lineId) =>
Areas.FirstOrDefault(a => a.LineIds.Contains(lineId, StringComparer.OrdinalIgnoreCase));
}
public sealed record UnsAreaSummary(string AreaId, string Name, IReadOnlyList<string> LineIds);
public sealed record UnsLineSummary(string LineId, string Name, int EquipmentCount, int TagCount);
/// <summary>
/// Opaque per-draft revision fingerprint. Preview fetches the current token + stores it
/// in the <see cref="UnsImpactPreview.RevisionToken"/>. Confirm compares the token against
/// the draft's live value; mismatch means another operator mutated the draft between
/// preview + commit — raise <c>409 Conflict / refresh-required</c> in the UI.
/// </summary>
public sealed record DraftRevisionToken(string Value)
{
/// <summary>Compare two tokens for equality; null-safe.</summary>
public bool Matches(DraftRevisionToken? other) =>
other is not null &&
string.Equals(Value, other.Value, StringComparison.Ordinal);
}
/// <summary>Output of <see cref="UnsImpactAnalyzer.Analyze"/>.</summary>
public sealed class UnsImpactPreview
{
public required int AffectedEquipmentCount { get; init; }
public required int AffectedTagCount { get; init; }
public required IReadOnlyList<string> CascadeWarnings { get; init; }
public required DraftRevisionToken RevisionToken { get; init; }
public required string HumanReadableSummary { get; init; }
}
/// <summary>Thrown when a move targets a different cluster than the source (decision #82).</summary>
public sealed class CrossClusterMoveRejectedException(string message) : Exception(message);
/// <summary>Thrown when the move operation references a source / target that doesn't exist in the draft.</summary>
public sealed class UnsMoveValidationException(string message) : Exception(message);