fix(tagconfig): close the four #551 residuals — accept-gate-first Inspect, three authoring rules, CSV/service gate, doc sweep

Gitea #551.

1. CSV import ran no driver-typed config gate. Its typed columns were always
   strictly parsed, but the free-form TagConfigJson column reached the database
   unread, so a CSV could author a tag the driver rejects — one that deploys
   green and reads BadNodeIdUnknown forever. RawTagCsvMapper.Parse now names the
   offending row in the staged review grid, and RawTreeService enforces the same
   gate on CreateTag/UpdateTag/ImportTags: a form cannot be the guarantee (the
   API and any future caller reach none of them, and this repo has no bUnit).
   A rejected import row is an ERROR — the batch inserts nothing and the row is
   named — never a skip.

   Note the brief's premise that manual entry rides ImportTagsAsync is wrong:
   RawManualTagEntryModal commits row-by-row via CreateTagAsync, which is why
   the gate had to land on all three write paths.

2/3. The seven pre-#549 Inspect methods ran a hand-list of DescribeInvalidEnum
   checks, so every rejection cause not on the list was invisible. They are now
   ACCEPT-GATE-FIRST (the MTConnect/Sql/OpcUaClient/Calculation shape): ask the
   driver's own reader for the verdict, then explain the refusal, with a
   catch-all so "the parse rejects but Inspect said nothing" is unreachable.
   That closes S7's absent dataType, AbCip members[], Modbus stringByteOrder and
   the type-dependent guards, plus causes the issue did not name (absent
   region/address/tagPath/symbolPath/topic, TwinCAT Structure, MQTT's Sparkplug
   tuple). FOCAS and MQTT keep advisories about ACCEPTED blobs, so they
   diagnose rather than return empty.

   Authoring half: ModbusTagConfigModel gains stringByteOrder;
   AbCipTagConfigModel asks the new AbCipTagDefinitionFactory.DescribeInvalidMember;
   S7TagConfigModel captures whether the SOURCE blob declared dataType and asks
   S7TagDefinitionFactory.DataTypeKey/DeclaresDataType. One authority per rule.

4. Doc sweep: the six factory Inspect summaries, EquipmentTagConfigInspector's
   class remarks and AdminOperationsActor's comment all claimed invalid enums are
   "silently defaulted by the lenient runtime" — false since R2-11 Phase C made
   the runtime reads strict. They describe rejections now.
This commit is contained in:
Joseph Doherty
2026-08-10 00:08:06 -04:00
parent b0d2fa30e3
commit 05b424788f
24 changed files with 1846 additions and 100 deletions
@@ -19,6 +19,14 @@ namespace ZB.MOM.WW.OtOpcUa.AdminUI.Uns;
/// columns. A typed column therefore <b>wins</b> over a conflicting key carried in the <c>TagConfigJson</c>
/// fallback; the review model flags when that override happened so the operator sees it.
/// </para>
/// <para>
/// The assembled blob is then put to the driver's own <see cref="TagConfigValidator"/> (Gitea #551): the
/// typed columns were always strictly parsed, but the free-form <c>TagConfigJson</c> column reached the
/// database unread, so a CSV could author a tag the driver rejects — one that deploys clean and reads
/// <c>BadNodeIdUnknown</c> forever. A failing row is an <b>error</b>, blocking the all-or-nothing commit;
/// it is never silently skipped. This check is for the operator's benefit (it names the row before
/// commit); the guarantee lives in <see cref="IRawTreeService.ImportTagsAsync"/>, which re-runs it.
/// </para>
/// </summary>
public static class RawTagCsvMapper
{
@@ -84,7 +92,7 @@ public static class RawTagCsvMapper
continue;
}
reviewed.Add(BuildRow(r, headers, headerIndex, cells, map, groupPathPrefix));
reviewed.Add(BuildRow(r, headers, headerIndex, cells, driverType, map, groupPathPrefix));
}
FlagDuplicates(reviewed);
@@ -96,6 +104,7 @@ public static class RawTagCsvMapper
string[] headers,
IReadOnlyDictionary<string, int> headerIndex,
string[] cells,
string driverType,
CsvDriverTagMap? map,
string? groupPathPrefix)
{
@@ -193,6 +202,19 @@ public static class RawTagCsvMapper
// Alarm flags — an alarm is authored iff Alarm.AlarmType is present.
json = ApplyAlarm(json, Cell("Alarm.AlarmType"), Cell("Alarm.Severity"), Cell("Alarm.HistorizeToAveva"));
// The driver-typed config gate, on the FINAL assembled blob (Gitea #551 item 1). Until this,
// the typed columns above were strictly parsed but the free-form TagConfigJson column was not
// checked at all, so a CSV could import a blob the driver rejects — a tag that deploys clean
// and then reads BadNodeIdUnknown forever. Surfacing it here, on the staged review row, is what
// lets the operator SEE which row is wrong before committing; IRawTreeService.ImportTagsAsync
// re-checks it as the actual guarantee (a form/grid cannot cover the API or a future caller).
// A rejected row is an ERROR, never a skip: the commit is all-or-nothing, and "skipped" would
// claim the tag is already present.
if (TagConfigValidator.Validate(driverType, json) is { } configError)
{
return Invalid(rowNumber, cells, $"TagConfig: {configError}");
}
var importRow = new RawTagImportRow(groupPath, new RawTagInput(name, dataType, access, writeIdempotent, pollGroupId, json));
return new RawCsvReviewRow
@@ -1,5 +1,6 @@
using System.Text.Json;
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors;
using ZB.MOM.WW.OtOpcUa.Commons.Types;
using ZB.MOM.WW.OtOpcUa.Configuration;
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
@@ -834,6 +835,9 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
t => t.DeviceId == deviceId && t.TagGroupId == tagGroupId && t.Name == input.Name, ct))
return new UnsMutationResult(false, $"A tag named '{input.Name}' already exists here.");
if (await ValidateTagConfigAsync(db, deviceId, tagConfig, ct) is { } configError)
return new UnsMutationResult(false, configError);
var id = NewId("TAG");
db.Tags.Add(new Tag
{
@@ -872,6 +876,9 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
&& t.Name == input.Name && t.TagId != tagId, ct))
return new UnsMutationResult(false, $"A tag named '{input.Name}' already exists here.");
if (await ValidateTagConfigAsync(db, entity.DeviceId, tagConfig, ct) is { } configError)
return new UnsMutationResult(false, configError);
db.Entry(entity).Property(e => e.RowVersion).OriginalValue = rowVersion;
entity.Name = input.Name;
entity.DataType = input.DataType;
@@ -902,6 +909,10 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
if (!await db.Devices.AnyAsync(dev => dev.DeviceId == deviceId, ct))
return new RawTagImportResult(0, new[] { $"Device '{deviceId}' not found." });
// The device's driver type, resolved ONCE: an import is per-device, and a device hangs off exactly
// one driver, so every row in the batch is judged by the same driver's rules (Gitea #551 item 1).
var driverType = await ResolveDriverTypeAsync(db, deviceId, ct);
// Existing groups on the device: a (parent-group-id | null, name) → group-id lookup we extend with
// pending (auto-created) groups so a repeated path within the batch resolves to one group.
var groupLookup = (await db.TagGroups.AsNoTracking()
@@ -951,6 +962,15 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
continue;
}
// The driver-typed config gate. A rejected row is an ERROR — the import is all-or-nothing, so
// the batch inserts nothing and the row is NAMED; it is never skipped ("skipped" would claim
// the tag is already present). See ValidateTagConfigAsync for why this lives in the service.
if (TagConfigValidator.Validate(driverType, tagConfig) is { } configError)
{
errors.Add($"{rowLabel}: TagConfig: {configError}");
continue;
}
if (!takenNames.Add((groupId, row.Tag.Name)))
{
errors.Add($"{rowLabel}: a tag named '{row.Tag.Name}' already exists at that path.");
@@ -1196,6 +1216,55 @@ public sealed class RawTreeService(IDbContextFactory<OtOpcUaConfigDbContext> dbF
}
}
/// <summary>
/// The driver-type string of the driver a device hangs off, or <see langword="null"/> when the
/// device (or its driver) is gone. Joined in exactly as <c>LoadDeviceForEditAsync</c> does — a
/// device carries no driver type of its own.
/// </summary>
/// <param name="db">The open context.</param>
/// <param name="deviceId">The device whose driver type to resolve.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The canonical driver-type string, or null.</returns>
private static Task<string?> ResolveDriverTypeAsync(
OtOpcUaConfigDbContext db, string deviceId, CancellationToken ct)
=> db.Devices.AsNoTracking()
.Where(dev => dev.DeviceId == deviceId)
.Join(db.DriverInstances.AsNoTracking(),
dev => dev.DriverInstanceId,
drv => drv.DriverInstanceId,
(dev, drv) => drv.DriverType)
.FirstOrDefaultAsync(ct);
/// <summary>
/// The driver-typed <c>TagConfig</c> gate, applied on the SERVER for every tag-writing path —
/// create, update and bulk import alike (Gitea #551 item 1).
/// <para>
/// <b>It lives here rather than in the modals because a form cannot be the guarantee.</b> The
/// tag modal and the manual-entry grid already run <see cref="TagConfigValidator"/> for fast
/// feedback, but the CSV import did not, and the API and any future caller reach none of them —
/// so a blob the driver rejects could be persisted, deploy clean, and read
/// <c>BadNodeIdUnknown</c> forever. This repo also has no bUnit, so a rule only a page enforces
/// is a rule no test can reach.
/// </para>
/// <para>
/// The rule itself is not stated here: <see cref="TagConfigValidator"/> dispatches to the
/// driver's typed model, which in turn asks the driver's own reader. An unmapped driver type
/// (Galaxy) validates to null, exactly as it does in the modals.
/// </para>
/// </summary>
/// <param name="db">The open context.</param>
/// <param name="deviceId">The device the tag belongs to (supplies the driver type).</param>
/// <param name="tagConfig">The normalised TagConfig JSON about to be persisted.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>An operator-facing error, or <see langword="null"/> when the config is acceptable.</returns>
private static async Task<string?> ValidateTagConfigAsync(
OtOpcUaConfigDbContext db, string deviceId, string tagConfig, CancellationToken ct)
{
var driverType = await ResolveDriverTypeAsync(db, deviceId, ct);
var error = TagConfigValidator.Validate(driverType, tagConfig);
return error is null ? null : $"TagConfig: {error}";
}
/// <summary>Ordinal (case-sensitive) equality for a nullable-parent + name tag-group key.</summary>
private sealed class GroupKeyComparer : IEqualityComparer<(string? Parent, string Name)>
{
@@ -59,11 +59,23 @@ public sealed class AbCipTagConfigModel
/// (<c>BadNodeIdUnknown</c>). Without this the blob would be accepted here, silently rewritten to
/// <c>DInt</c> on save, and — on the CSV-import and API paths, which save it verbatim — deployed
/// into a tag the driver skips (Gitea #549).
/// <para>
/// <b>Declared UDT members are checked too (Gitea #551), and the rule is not restated here.</b>
/// The member rules are as strict as the top-level ones — one member with no <c>name</c> or a
/// typo'd <c>dataType</c> rejects the WHOLE tag — but nothing on the authoring side saw them:
/// this editor emits only <c>tagPath</c>/<c>dataType</c>/<c>writable</c>, so <c>members</c>
/// rides <see cref="_bag"/> untouched from whatever hand-authored / CSV / API blob produced it.
/// The verdict AND the wording come from
/// <see cref="AbCipTagDefinitionFactory.DescribeInvalidMember(string)"/>, the same authority the
/// driver's reader and the deploy gate consult, so this gate cannot drift from the runtime in
/// either direction.
/// </para>
/// </remarks>
/// <returns>An error message describing the validation failure, or <c>null</c> when valid.</returns>
public string? Validate()
{
if (TagConfigJson.DescribeInvalidEnum<AbCipDataType>(_bag, "dataType") is { } dtError) { return dtError; }
return string.IsNullOrWhiteSpace(TagPath) ? "Tag path is required." : null;
if (string.IsNullOrWhiteSpace(TagPath)) { return "Tag path is required."; }
return AbCipTagDefinitionFactory.DescribeInvalidMember(TagConfigJson.Serialize(_bag));
}
}
@@ -69,17 +69,26 @@ public sealed class ModbusTagConfigModel
/// <summary>Validation hook; returns an error message or null when the model is valid.</summary>
/// <remarks>
/// All three enums are read from the ORIGINAL key bag, not from the leniently-defaulted typed
/// All FOUR enums are read from the ORIGINAL key bag, not from the leniently-defaulted typed
/// fields above: <c>ModbusTagDefinitionFactory</c> reads each with <c>TryReadEnumStrict</c>, so a
/// present-but-invalid string rejects the whole tag at the driver (<c>BadNodeIdUnknown</c>) where
/// the load silently substitutes <c>HoldingRegisters</c>/<c>Int16</c>/<c>BigEndian</c>. Accepting
/// one here would let the CSV-import and API paths — which save the blob verbatim — deploy a tag
/// the driver skips (Gitea #549). Also note a <b>wrong-but-valid</b> region is not this gate's
/// business: only a spelling the runtime cannot read is rejected.
/// the load silently substitutes <c>HoldingRegisters</c>/<c>Int16</c>/<c>BigEndian</c>/
/// <c>HighByteFirst</c>. Accepting one here would let the CSV-import and API paths — which save the
/// blob verbatim — deploy a tag the driver skips (Gitea #549). Also note a <b>wrong-but-valid</b>
/// region is not this gate's business: only a spelling the runtime cannot read is rejected.
/// <para>
/// <c>stringByteOrder</c> is the fourth (Gitea #551): the runtime has always read it as
/// strictly as its three siblings, but this model — and the deploy gate's <c>Inspect</c> —
/// checked only the other three, so a typo'd <c>stringByteOrder</c> passed BOTH gates and died
/// at the driver. It is not in <see cref="ToJson"/>'s exposed-field list either, so it can only
/// ever reach the bag from a hand-authored blob, a CSV import or the API — exactly the paths
/// this gate exists for.
/// </para>
/// </remarks>
/// <returns>An error message describing the validation failure, or <c>null</c> when the model is valid.</returns>
public string? Validate()
=> TagConfigJson.DescribeInvalidEnum<ModbusRegion>(_bag, "region")
?? TagConfigJson.DescribeInvalidEnum<ModbusDataType>(_bag, "dataType")
?? TagConfigJson.DescribeInvalidEnum<ModbusByteOrder>(_bag, "byteOrder");
?? TagConfigJson.DescribeInvalidEnum<ModbusByteOrder>(_bag, "byteOrder")
?? TagConfigJson.DescribeInvalidEnum<ModbusStringByteOrder>(_bag, "stringByteOrder");
}
@@ -22,6 +22,19 @@ public sealed class S7TagConfigModel
private JsonObject _bag = new();
// Whether the blob this model was LOADED FROM was missing the "dataType" key entirely — captured
// because the lenient load above cannot tell "absent" from "present": both land on Int16. S7 is the one
// driver whose reader REQUIRES the key (S7TagDefinitionFactory.DeclaresDataType), so a blob without one
// is rejected outright and reads BadNodeIdUnknown; without this capture that blob validated clean
// (Gitea #551). Same capture-the-as-loaded-shape pattern as MTConnectTagConfigModel's _rawDataType.
//
// It records a property of the SOURCE, so the default (false) is the honest answer for a model built by
// object initialiser rather than loaded — RawBrowseCommitMapper does exactly that, and ToJson() then
// writes the key from the typed DataType field. ToJson() likewise clears it: a blob that has been
// re-serialised through this model always carries the key, so it can only ever be true on the FIRST
// load of a blob authored elsewhere (hand-edited, CSV import, API).
private bool _dataTypeMissingInSource;
/// <summary>Loads a model from a TagConfig JSON string, defaulting any absent field and retaining
/// every original key (so fields this editor doesn't expose survive a load→save).</summary>
/// <param name="json">The raw TagConfig JSON string, or null/empty for a fresh model.</param>
@@ -32,9 +45,10 @@ public sealed class S7TagConfigModel
return new S7TagConfigModel
{
Address = TagConfigJson.GetString(o, "address") ?? "",
DataType = TagConfigJson.GetEnum(o, "dataType", S7DataType.Int16),
DataType = TagConfigJson.GetEnum(o, S7TagDefinitionFactory.DataTypeKey, S7DataType.Int16),
StringLength = TagConfigJson.GetInt(o, "stringLength"),
Writable = TagConfigJson.GetBoolNullable(o, "writable"),
_dataTypeMissingInSource = !o.ContainsKey(S7TagDefinitionFactory.DataTypeKey),
_bag = o,
};
}
@@ -45,7 +59,8 @@ public sealed class S7TagConfigModel
public string ToJson()
{
TagConfigJson.Set(_bag, "address", Address.Trim());
TagConfigJson.Set(_bag, "dataType", DataType);
TagConfigJson.Set(_bag, S7TagDefinitionFactory.DataTypeKey, DataType);
_dataTypeMissingInSource = false;
TagConfigJson.Set(_bag, "stringLength", StringLength);
TagConfigJson.Set(_bag, "writable", Writable);
return TagConfigJson.Serialize(_bag);
@@ -59,10 +74,36 @@ public sealed class S7TagConfigModel
/// so a present-but-invalid string rejects the whole tag at the driver (<c>BadNodeIdUnknown</c>)
/// where the load silently substitutes <c>Int16</c>. Accepting it here would let the CSV-import
/// and API paths — which save the blob verbatim — deploy an unreadable tag (Gitea #549).
/// <para>
/// <b>An ABSENT <c>dataType</c> is rejected too (Gitea #551), and S7 is the only driver where
/// that is right.</b> <c>S7TagDefinitionFactory.FromTagConfig</c> refuses a blob that does not
/// carry the key at all — unique in the fleet, where every sibling falls back to a default — so
/// such a tag deployed clean through both gates and then read <c>BadNodeIdUnknown</c> forever.
/// Presence is judged on the key, not the value, matching
/// <see cref="S7TagDefinitionFactory.DeclaresDataType"/>: a non-string token is present-enough
/// for the reader, which then treats it as absent and accepts the tag as <c>Int16</c>.
/// </para>
/// <para>
/// <b>The one blob this can newly block in the modal is one that was already dead.</b> The
/// typed editor pushes <see cref="ToJson"/> — which always writes <c>dataType</c> — only when a
/// field CHANGES, so opening a legacy key-less blob and saving it untouched hands this method
/// the as-loaded string. That save is refused, and correctly: the tag it would persist is one
/// the driver already skips. Picking any data type in the editor writes the key and clears it.
/// </para>
/// </remarks>
public string? Validate()
{
if (TagConfigJson.DescribeInvalidEnum<S7DataType>(_bag, "dataType") is { } dtError) { return dtError; }
return string.IsNullOrWhiteSpace(Address) ? "Address is required." : null;
if (TagConfigJson.DescribeInvalidEnum<S7DataType>(_bag, S7TagDefinitionFactory.DataTypeKey) is { } dtError)
{
return dtError;
}
if (string.IsNullOrWhiteSpace(Address)) { return "Address is required."; }
return _dataTypeMissingInSource
? "A data type is required — the S7 driver rejects a tag whose TagConfig carries no "
+ $"'{S7TagDefinitionFactory.DataTypeKey}' key (it would read BadNodeIdUnknown). Valid: "
+ string.Join(", ", Enum.GetNames<S7DataType>()) + "."
: null;
}
}
@@ -246,8 +246,11 @@ public sealed class AdminOperationsActor : ReceiveActor
}
// R2-11 (05/CONV-2): deploy-time TagConfig strictness inspection over equipment tags. The
// driver-typed EquipmentTagConfigInspector surfaces present-but-invalid enum values (which the
// lenient runtime silently defaults) + structurally-unparseable TagConfig. In Warn mode
// driver-typed EquipmentTagConfigInspector surfaces every blob the driver's own reader would
// REJECT — a present-but-invalid enum value, an absent or malformed address field, a
// structurally-unparseable TagConfig — each of which makes the driver skip the tag so it reads
// BadNodeIdUnknown forever. (Pre-R2-11 Phase C the runtime silently defaulted a typo'd enum;
// it has rejected the tag since, so these warnings describe rejections.) In Warn mode
// (default) these are non-blocking (logged + appended to the accepted result Message); in Error
// mode they fold into the reject list so a typo'd config cannot be re-deployed until fixed.
// Running servers are untouched either way — the gate only sees re-deploys.
@@ -17,8 +17,17 @@ namespace ZB.MOM.WW.OtOpcUa.ControlPlane.AdminOperations;
/// Deploy-time TagConfig inspection dispatched by <c>DriverType</c> (R2-11, 05/CONV-2). Mirrors the
/// AdminUI <c>TagConfigValidator</c> shape: a case-insensitive map from the canonical driver-type string
/// to that driver's lightweight <c>Contracts</c> parser <c>Inspect()</c>, returning human-readable
/// warnings (present-but-invalid enum values that the lenient runtime silently defaults, plus
/// structurally-unparseable TagConfig). Never throws.
/// warnings. Never throws.
/// <para><b>The warnings describe REJECTIONS, not silent defaults.</b> Every driver's runtime read has
/// been strict since R2-11 Phase C (<c>TagConfigJson.TryReadEnumStrict</c>): a present-but-invalid enum
/// value — like an absent or malformed address field, or an unparseable document — makes the reader
/// refuse the whole tag, so the driver skips it and it reads <c>BadNodeIdUnknown</c> forever with the
/// deployment otherwise green. That is what these warnings are for. (Before Phase C the runtime did
/// silently substitute a default for a typo'd enum; the doc-comments here and on the per-driver
/// <c>Inspect</c> methods still said so until Gitea #551.) Because a warning can be PROMOTED to a
/// blocking error by <c>TagConfigValidationMode.Error</c>, each <c>Inspect</c> asks its driver's own
/// reader for the verdict and only explains a refusal — never restating the rule, and so never able to
/// be stricter than the runtime it reports on.</para>
/// <para><b>Coverage is every driver that ships an <c>Inspect</c>, and that invariant is guarded</b> by
/// <c>EquipmentTagConfigInspectorCoverageTests</c>, which reflects over the referenced
/// <c>*.Contracts</c> assemblies — so a driver that gains an inspector cannot be silently left out. This