dohertj2/lmxopcua
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 6.4 Stream A + B data layer — UnsImpactAnalyzer + EquipmentCsvImporter (parser)

Browse Source 
Ships the pure-logic data layer of Phase 6.4. Blazor UI pieces
(UnsTab drag/drop, CSV import modal, preview table, FinaliseImportBatch txn,
staging tables) are deferred to visual-compliance follow-ups (tasks #153,
#155, #157).

Admin.Services additions:

- UnsImpactAnalyzer.Analyze(snapshot, move) — pure-function, no I/O. Three
  move variants: LineMove, AreaRename, LineMerge. Returns UnsImpactPreview
  with AffectedEquipmentCount + AffectedTagCount + CascadeWarnings +
  RevisionToken + HumanReadableSummary the Admin UI shows in the confirm
  modal. Cross-cluster moves rejected with CrossClusterMoveRejectedException
  per decision #82. Missing source/target throws UnsMoveValidationException.
  Surfaces sibling-line same-name ambiguity as a cascade warning.
- DraftRevisionToken — opaque revision fingerprint. Preview captures the
  token; Confirm compares it. The 409-concurrent-edit UX plumbs through on
  the Razor-page follow-up (task #153). Matches(other) is null-safe.
- UnsTreeSnapshot + UnsAreaSummary + UnsLineSummary — snapshot shape the
  caller hands to the analyzer. Tests build them in-memory without a DB.

- EquipmentCsvImporter.Parse(csvText) — RFC 4180 CSV parser per decision #95.
  Version-marker contract: line 1 must be "# OtOpcUaCsv v1" (future shapes
  bump the version). Required columns from decision #117 + optional columns
  from decision #139. Rejects unknown columns, duplicate column names,
  blank required fields, duplicate ZTags within the file. Quoted-field
  handling supports embedded commas + escaped "" quotes. Returns
  EquipmentCsvParseResult { AcceptedRows, RejectedRows } so the preview
  modal renders accept/reject counts without re-parsing.

Tests (22 new, all pass):

- UnsImpactAnalyzerTests (9): line move counts equipment + tags; cross-
  cluster throws; unknown source/target throws validation; ambiguous same-
  name target raises warning; area rename sums across lines; line merge
  cross-area warns; same-area merge no warning; DraftRevisionToken matches
  semantics.
- EquipmentCsvImporterTests (13): empty file throws; missing version marker;
  missing required column; unknown column; duplicate column; valid single
  row round-trips; optional columns populate when present; blank required
  field rejects row; duplicate ZTag rejects second; RFC 4180 quoted fields
  with commas + escaped quotes; mismatched column count rejects; blank
  lines between rows ignored; required + optional column constants match
  decisions #117 + #139 exactly.

Full solution dotnet test: 1159 passing (Phase 6.3 = 1137, Phase 6.4 A+B
data = +22). Pre-existing Client.CLI Subscribe flake unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-04-19 10:09:47 -04:00
parent 4901b78e9a
commit 560a961cca
4 changed files with 814 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

259
src/ZB.MOM.WW.OtOpcUa.Admin/Services/EquipmentCsvImporter.cs Normal file
View File

@@ -0,0 +1,259 @@
using System.Globalization;
using System.Text;
namespace ZB.MOM.WW.OtOpcUa.Admin.Services;
/// <summary>
/// RFC 4180 CSV parser for equipment import per decision #95 and Phase 6.4 Stream B.1.
/// Produces a validated <see cref="EquipmentCsvParseResult"/> the caller (CSV import
/// modal + staging tables) consumes. Pure-parser concern — no DB access, no staging
/// writes; those live in the follow-up Stream B.2 work.
/// </summary>
/// <remarks>
/// <para><b>Header contract</b>: line 1 must be exactly <c># OtOpcUaCsv v1</c> (version
/// marker). Line 2 is the column header row. Unknown columns are rejected; required
/// columns must all be present. The version bump handshake lets future shapes parse
/// without ambiguity — v2 files go through a different parser variant.</para>
///
/// <para><b>Required columns</b> per decision #117: ZTag, MachineCode, SAPID,
/// EquipmentId, EquipmentUuid, Name, UnsAreaName, UnsLineName.</para>
///
/// <para><b>Optional columns</b> per decision #139: Manufacturer, Model, SerialNumber,
/// HardwareRevision, SoftwareRevision, YearOfConstruction, AssetLocation,
/// ManufacturerUri, DeviceManualUri.</para>
///
/// <para><b>Row validation</b>: blank required field → rejected; duplicate ZTag within
/// the same file → rejected. Duplicate against the DB isn't detected here — the
/// staged-import finalize step (Stream B.4) catches that.</para>
/// </remarks>
public static class EquipmentCsvImporter
{
public const string VersionMarker = "# OtOpcUaCsv v1";
public static IReadOnlyList<string> RequiredColumns { get; } = new[]
{
"ZTag", "MachineCode", "SAPID", "EquipmentId", "EquipmentUuid",
"Name", "UnsAreaName", "UnsLineName",
};
public static IReadOnlyList<string> OptionalColumns { get; } = new[]
{
"Manufacturer", "Model", "SerialNumber", "HardwareRevision", "SoftwareRevision",
"YearOfConstruction", "AssetLocation", "ManufacturerUri", "DeviceManualUri",
};
public static EquipmentCsvParseResult Parse(string csvText)
{
ArgumentNullException.ThrowIfNull(csvText);
var rows = SplitLines(csvText);
if (rows.Count == 0)
throw new InvalidCsvFormatException("CSV is empty.");
if (!string.Equals(rows[0].Trim(), VersionMarker, StringComparison.Ordinal))
throw new InvalidCsvFormatException(
$"CSV header line 1 must be exactly '{VersionMarker}' — got '{rows[0]}'. " +
"Files without the version marker are rejected so future-format files don't parse ambiguously.");
if (rows.Count < 2)
throw new InvalidCsvFormatException("CSV has no column header row (line 2) or data rows.");
var headerCells = SplitCsvRow(rows[1]);
ValidateHeader(headerCells);
var accepted = new List<EquipmentCsvRow>();
var rejected = new List<EquipmentCsvRowError>();
var ztagsSeen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var colIndex = headerCells
.Select((name, idx) => (name, idx))
.ToDictionary(t => t.name, t => t.idx, StringComparer.OrdinalIgnoreCase);
for (var i = 2; i < rows.Count; i++)
{
if (string.IsNullOrWhiteSpace(rows[i])) continue;
try
{
var cells = SplitCsvRow(rows[i]);
if (cells.Length != headerCells.Length)
{
rejected.Add(new EquipmentCsvRowError(
LineNumber: i + 1,
Reason: $"Column count {cells.Length} != header count {headerCells.Length}."));
continue;
}
var row = BuildRow(cells, colIndex);
var missing = RequiredColumns.Where(c => string.IsNullOrWhiteSpace(GetCell(row, c))).ToList();
if (missing.Count > 0)
{
rejected.Add(new EquipmentCsvRowError(i + 1, $"Blank required column(s): {string.Join(", ", missing)}"));
continue;
}
if (!ztagsSeen.Add(row.ZTag))
{
rejected.Add(new EquipmentCsvRowError(i + 1, $"Duplicate ZTag '{row.ZTag}' within file."));
continue;
}
accepted.Add(row);
}
catch (InvalidCsvFormatException ex)
{
rejected.Add(new EquipmentCsvRowError(i + 1, ex.Message));
}
}
return new EquipmentCsvParseResult(accepted, rejected);
}
private static void ValidateHeader(string[] headerCells)
{
var seen = new HashSet<string>(headerCells, StringComparer.OrdinalIgnoreCase);
// Missing required
var missingRequired = RequiredColumns.Where(r => !seen.Contains(r)).ToList();
if (missingRequired.Count > 0)
throw new InvalidCsvFormatException($"Header is missing required column(s): {string.Join(", ", missingRequired)}");
// Unknown columns (not in required ∪ optional)
var known = new HashSet<string>(RequiredColumns.Concat(OptionalColumns), StringComparer.OrdinalIgnoreCase);
var unknown = headerCells.Where(c => !known.Contains(c)).ToList();
if (unknown.Count > 0)
throw new InvalidCsvFormatException(
$"Header has unknown column(s): {string.Join(", ", unknown)}. " +
"Bump the version marker to define a new shape before adding columns.");
// Duplicates
var dupe = headerCells.GroupBy(c => c, StringComparer.OrdinalIgnoreCase).FirstOrDefault(g => g.Count() > 1);
if (dupe is not null)
throw new InvalidCsvFormatException($"Header has duplicate column '{dupe.Key}'.");
}
private static EquipmentCsvRow BuildRow(string[] cells, Dictionary<string, int> colIndex) => new()
{
ZTag = cells[colIndex["ZTag"]],
MachineCode = cells[colIndex["MachineCode"]],
SAPID = cells[colIndex["SAPID"]],
EquipmentId = cells[colIndex["EquipmentId"]],
EquipmentUuid = cells[colIndex["EquipmentUuid"]],
Name = cells[colIndex["Name"]],
UnsAreaName = cells[colIndex["UnsAreaName"]],
UnsLineName = cells[colIndex["UnsLineName"]],
Manufacturer = colIndex.TryGetValue("Manufacturer", out var mi) ? cells[mi] : null,
Model = colIndex.TryGetValue("Model", out var moi) ? cells[moi] : null,
SerialNumber = colIndex.TryGetValue("SerialNumber", out var si) ? cells[si] : null,
HardwareRevision = colIndex.TryGetValue("HardwareRevision", out var hi) ? cells[hi] : null,
SoftwareRevision = colIndex.TryGetValue("SoftwareRevision", out var swi) ? cells[swi] : null,
YearOfConstruction = colIndex.TryGetValue("YearOfConstruction", out var yi) ? cells[yi] : null,
AssetLocation = colIndex.TryGetValue("AssetLocation", out var ai) ? cells[ai] : null,
ManufacturerUri = colIndex.TryGetValue("ManufacturerUri", out var mui) ? cells[mui] : null,
DeviceManualUri = colIndex.TryGetValue("DeviceManualUri", out var dui) ? cells[dui] : null,
};
private static string GetCell(EquipmentCsvRow row, string colName) => colName switch
{
"ZTag" => row.ZTag,
"MachineCode" => row.MachineCode,
"SAPID" => row.SAPID,
"EquipmentId" => row.EquipmentId,
"EquipmentUuid" => row.EquipmentUuid,
"Name" => row.Name,
"UnsAreaName" => row.UnsAreaName,
"UnsLineName" => row.UnsLineName,
_ => string.Empty,
};
/// <summary>Split the raw text on line boundaries. Handles \r\n + \n + \r.</summary>
private static List<string> SplitLines(string csv) =>
csv.Split(["\r\n", "\n", "\r"], StringSplitOptions.None).ToList();
/// <summary>Split one CSV row with RFC 4180 quoted-field handling.</summary>
private static string[] SplitCsvRow(string row)
{
var cells = new List<string>();
var sb = new StringBuilder();
var inQuotes = false;
for (var i = 0; i < row.Length; i++)
{
var ch = row[i];
if (inQuotes)
{
if (ch == '"')
{
// Escaped quote "" inside quoted field.
if (i + 1 < row.Length && row[i + 1] == '"')
{
sb.Append('"');
i++;
}
else
{
inQuotes = false;
}
}
else
{
sb.Append(ch);
}
}
else
{
if (ch == ',')
{
cells.Add(sb.ToString());
sb.Clear();
}
else if (ch == '"' && sb.Length == 0)
{
inQuotes = true;
}
else
{
sb.Append(ch);
}
}
}
cells.Add(sb.ToString());
return cells.ToArray();
}
}
/// <summary>One parsed equipment row with required + optional fields.</summary>
public sealed class EquipmentCsvRow
{
// Required (decision #117)
public required string ZTag { get; init; }
public required string MachineCode { get; init; }
public required string SAPID { get; init; }
public required string EquipmentId { get; init; }
public required string EquipmentUuid { get; init; }
public required string Name { get; init; }
public required string UnsAreaName { get; init; }
public required string UnsLineName { get; init; }
// Optional (decision #139 — OPC 40010 Identification fields)
public string? Manufacturer { get; init; }
public string? Model { get; init; }
public string? SerialNumber { get; init; }
public string? HardwareRevision { get; init; }
public string? SoftwareRevision { get; init; }
public string? YearOfConstruction { get; init; }
public string? AssetLocation { get; init; }
public string? ManufacturerUri { get; init; }
public string? DeviceManualUri { get; init; }
}
/// <summary>One row-level rejection captured by the parser. Line-number is 1-based in the source file.</summary>
public sealed record EquipmentCsvRowError(int LineNumber, string Reason);
/// <summary>Parser output — accepted rows land in staging; rejected rows surface in the preview modal.</summary>
public sealed record EquipmentCsvParseResult(
IReadOnlyList<EquipmentCsvRow> AcceptedRows,
IReadOnlyList<EquipmentCsvRowError> RejectedRows);
/// <summary>Thrown for file-level format problems (missing version marker, bad header, etc.).</summary>
public sealed class InvalidCsvFormatException(string message) : Exception(message);