diff --git a/docs/plans/2026-06-16-native-typed-json-design.md b/docs/plans/2026-06-16-native-typed-json-design.md new file mode 100644 index 00000000..8d39fb0e --- /dev/null +++ b/docs/plans/2026-06-16-native-typed-json-design.md @@ -0,0 +1,78 @@ +# Native-Typed JSON for List Attribute Values — Design + +**Date:** 2026-06-16 +**Status:** Implemented (NJ-1 … NJ-6, branch `feature/native-typed-json`) — full solution builds 0/0; feature-targeted tests green across Commons, TemplateEngine, ConfigurationDatabase, SiteRuntime, and Transport. Follow-up **#93/M3** (populate `InstanceAttributeOverride.ElementDataType` on write) was folded into NJ-2 so the central normalizer can read the override element type directly. +**Branch:** `feature/native-typed-json` + +## Problem + +The multi-value (List) attribute feature (shipped 2026-06-16, branch `feature/multivalue-attribute`) stores List values via `AttributeValueCodec` as a JSON **array of strings** — e.g. an `Int32` list is `["10","20","30"]` and a `Boolean` list is `["True","False"]`. This is internally consistent and round-trips, but it is not "native-typed" JSON: numbers and booleans are quoted, and `DateTime` uses a US-invariant format rather than ISO-8601. We want the canonical form to be native-typed (`[10,20,30]`, `[true,false]`, ISO dates), while existing persisted data is normalized to the new form (no dual-format data left behind). + +## Decisions + +| Decision | Choice | +|---|---| +| Encode form | Native-typed JSON: numbers/bools unquoted, strings quoted, `DateTime` as ISO-8601 string | +| Decode | **Read both** old (array-of-strings) and new (native) forms — backward compatible | +| Existing data | **Migrate** to native form across MS SQL + site SQLite + on bundle import (Approach B, thorough) | +| MS SQL mechanism | Idempotent C# **startup normalizer** (not T-SQL — type-aware JSON re-emission is fragile in SQL) | +| Site SQLite mechanism | **Active** normalization in the InstanceActor override-load path (it already has the element type) | +| Bundles | Normalize **on import** (already-exported files are external/unreachable) | + +**Reality note:** the List feature shipped this session and was not deployed to the docker cluster, so there is almost certainly **zero** old-form List data in any store yet. The migration is a safety net guaranteeing no dual-format data ever lingers, not a fix for existing broken data. + +## Architecture + +### 1. Codec (`AttributeValueCodec`, Commons) — foundation + +- **`Encode`** — only the list branch changes. Instead of mapping each element to an invariant string then serializing, serialize the typed CLR collection directly (`JsonSerializer.Serialize(enumerable, …)`) so `System.Text.Json` emits each element in its native JSON kind. STJ numbers/bools are culture-invariant by spec; `DateTime` serializes as round-trippable ISO-8601. Scalars (the `string` / `IFormattable` branches) are untouched. +- **`Decode`** — read both forms. Deserialize to `JsonElement[]` (instead of `string?[]`); for each element feed `ParseScalar` either `GetString()` (JSON string element) or `GetRawText()` (number/bool element). So `[10,20]` and `["10","20"]` both decode to `List{10,20}`; ISO and old US-invariant `DateTime` strings both parse via the existing `DateTime.Parse(…, RoundtripKind)`. A JSON `null` element still throws `FormatException` ("elements may not be null"), unchanged. +- The read-both Decode is also what makes the migration idempotent: re-encoding an already-native value yields identical bytes. + +### 2. MS SQL — idempotent central startup normalizer + +A normalization step invoked once after `dbContext.Database.MigrateAsync(...)` in `MigrationHelper.ApplyOrValidateMigrationsAsync` (active central node only). For each List row: + +- **`TemplateAttributes`** where `DataType = 'List'`: read `Value` + the row's own `ElementDataType`; compute `Encode(Decode(value, List, elementType))`; if it differs from the stored string, `UPDATE`. +- **`InstanceAttributeOverrides`** for List attributes: these rows may have a null `ElementDataType` (it is currently informational — see follow-up #93/M3), so resolve the element type via the owning instance's template attribute (instance → `TemplateId` → `TemplateAttribute` by name → `ElementDataType`). Then re-encode as above. + +Idempotent (native→native is a no-op `UPDATE`-skip), so the step is safe to leave in permanently and cheap on every subsequent startup (a scan, no writes). Per-row failures (malformed JSON, unresolved element type) are logged and skipped — normalization NEVER aborts startup (mirrors the audit/best-effort principle). The scan is bounded to List rows only. + +### 3. Site SQLite — active normalization on override load + +Site static-override values (`SiteStorageService`) are keyed by `(instance, canonicalName)` and carry no element type — the element type lives in the instance's flattened config. The natural normalization point is therefore the **InstanceActor override-load path** (`HandleOverridesLoaded`, added in MV-7), which already decodes both forms using the `ResolvedAttribute`'s `ElementDataType`. Extend it so that when a List override's stored string is in old form (i.e. `Encode(decoded)` differs from the stored string), it re-persists the native form via `SiteStorageService.SetStaticOverrideAsync`. This actively normalizes on every instance load (site startup / failover), reuses the existing decode + element type, and is idempotent. + +### 4. Bundles — normalize on import + +Already-exported `.bundle` files are external artifacts we cannot reach to rewrite; import already reads both forms via the codec. To ensure imported List values land in native form in the DB, the importer re-encodes List attribute `Value`s through the codec when writing (and the MS SQL normalizer is a backstop on next startup). No file rewriting. + +## Error handling + +- Decode of a genuinely malformed value still throws `FormatException`; the normalizers catch it per-row, log, and skip (no startup abort, no actor crash). +- The codec change is additive on the wire (`gRPC` `string value` field unchanged; `List` is a new type with no external wire consumer relying on the old quoted form). + +## Testing + +- **Codec:** native-form encode per element type (`[10,20,30]`, `[true,false]`, ISO `DateTime`, strings stay `["a","b"]`); old-form backward-compat decode (`["10","20"]` → `List`); round-trip for every element type; malformed still throws; culture-invariance preserved. +- **MS SQL normalizer:** old-form row → rewritten to native; native row → untouched (idempotency); malformed row → skipped + logged, other rows still processed; override row element-type resolved via template attribute. +- **Site SQLite / InstanceActor:** an old-form List override on load → re-persisted native (assert `SetStaticOverrideAsync` called with native form); a native override → not re-persisted (idempotent); scalar overrides unaffected. +- **Bundle import:** importing an old-form bundle lands native-form Values in the DB. + +## Out of scope / follow-ups + +- **Deployed-config snapshot is a fourth, un-normalized List-value store (latent gap, I-1).** + `DeployedConfigSnapshot.ConfigurationJson` + `RevisionHash` freeze the flattened config at + deploy time. The staleness/diff path (`DeploymentService.GetDeploymentComparisonAsync` → + `DiffService.AttributesEqual` ordinal compare + `RevisionHashService` SHA over the raw + `Value`) compares that frozen blob against a freshly-flattened (now native-form) config. If a + List attribute was ever *deployed* in old-form, the snapshot stays old-form → a spurious + "Changed" diff + false staleness flag until redeployed. This **cannot fire against current + data** (no List attributes were ever deployed — see the Reality Note), so it is recorded as a + known latent gap, not fixed. If hardening is wanted before List attributes are deployed at + scale: route the deserialized snapshot's List values through `Decode→Encode` in + `GetDeploymentComparisonAsync` before the diff/hash (symmetric with the other normalizers). +- CLI `template attribute` help still illustrates `--value` with a quoted string-list example; + add a native-form numeric example (e.g. `[10,20]`) so users don't hand-author quoted numbers + that get silently re-normalized. Doc-only; the quoted form still decodes. +- Rewriting already-exported bundle files (unreachable). +- This pairs naturally with follow-up **#93/M3** (populate `InstanceAttributeOverride.ElementDataType` on write); if done, the override normalizer could read the column directly instead of joining to the template attribute. Not required here. diff --git a/docs/plans/2026-06-16-native-typed-json.md b/docs/plans/2026-06-16-native-typed-json.md new file mode 100644 index 00000000..4eb36023 --- /dev/null +++ b/docs/plans/2026-06-16-native-typed-json.md @@ -0,0 +1,282 @@ +# Native-Typed JSON for List Attributes — Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task. + +**Goal:** Make `AttributeValueCodec` encode List values as native-typed JSON (`[10,20]`, `[true,false]`, ISO dates) while decoding both old (array-of-strings) and new forms, and normalize already-persisted data to the native form across MS SQL, site SQLite, and bundle import. + +**Architecture:** Native `Encode` + read-both `Decode` in the codec is the foundation. An idempotent central MS SQL startup normalizer rewrites old-form rows; the InstanceActor re-persists native form on override-load (site SQLite); the bundle importer re-encodes on import. To give the central normalizer a reliable element type for instance-override rows, the override-write path now stamps `ElementDataType` (folds in follow-up #93/M3). + +**Tech Stack:** C#/.NET 10, System.Text.Json, EF Core 10 (MS SQL + SQLite), Akka.NET, Transport bundles. + +**Design doc:** `docs/plans/2026-06-16-native-typed-json-design.md` (approved). +**Branch:** `feature/native-typed-json` (off main; design committed `91b1aa1`). + +**Conventions for every task:** TDD (failing test → fail → implement → pass → commit). Targeted builds/tests only — `dotnet build src//.csproj`, `dotnet test tests//.csproj --filter `. Full-solution build only in the final task. Do NOT create git worktrees. Commit with pathspec form (`git commit -m "…" -- `) and retry on `.git/index.lock` when implementers run concurrently. + +--- + +### Task NJ-1: Codec — native-typed Encode + read-both Decode + +**Classification:** high-risk +**Estimated implement time:** ~5 min +**Parallelizable with:** NJ-2 +**Blocked by:** none (foundation) + +**Files:** +- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs` +- Modify: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs` + +This changes the canonical on-disk/on-wire form for List values. It is backward compatible (Decode reads both forms) and additive on the gRPC `string value` field (List is a new type with no external consumer of the old quoted form). + +**Step 1 — Update/extend tests (TDD).** Update the existing `Encode_DoubleList_IsInvariant` assertion to the native form, and add the native + backward-compat cases: + +```csharp +[Fact] +public void Encode_Int32List_ProducesNativeNumbers() => + Assert.Equal("[10,20,30]", AttributeValueCodec.Encode(new List { 10, 20, 30 })); + +[Fact] +public void Encode_BoolList_ProducesNativeBooleans() => + Assert.Equal("[true,false]", AttributeValueCodec.Encode(new List { true, false })); + +[Fact] +public void Encode_DoubleList_IsNativeAndInvariant() +{ + var original = CultureInfo.CurrentCulture; + try + { + CultureInfo.CurrentCulture = new CultureInfo("de-DE"); + Assert.Equal("[1.5,2.5]", AttributeValueCodec.Encode(new List { 1.5, 2.5 })); + } + finally { CultureInfo.CurrentCulture = original; } +} + +[Fact] +public void Encode_StringList_StaysQuoted() => + Assert.Equal("[\"a\",\"b\"]", AttributeValueCodec.Encode(new List { "a", "b" })); + +[Fact] +public void Encode_DateTimeList_IsIso8601() +{ + var json = AttributeValueCodec.Encode(new List { new(2026, 6, 16, 0, 0, 0, DateTimeKind.Utc) }); + Assert.Contains("2026-06-16T00:00:00", json); // ISO, not "06/16/2026 00:00:00" +} + +[Fact] +public void Decode_NewNativeIntForm_Parses() +{ + var back = (IList)AttributeValueCodec.Decode("[10,20]", DataType.List, DataType.Int32)!; + Assert.Equal(new[] { 10, 20 }, back); +} + +[Fact] +public void Decode_OldStringIntForm_BackwardCompatible() +{ + var back = (IList)AttributeValueCodec.Decode("[\"10\",\"20\"]", DataType.List, DataType.Int32)!; + Assert.Equal(new[] { 10, 20 }, back); +} + +[Theory] +[InlineData("[true,false]")] // new native bools +[InlineData("[\"True\",\"False\"]")] // old string bools +public void Decode_BoolForms_BothParse(string json) +{ + var back = (IList)AttributeValueCodec.Decode(json, DataType.List, DataType.Boolean)!; + Assert.Equal(new[] { true, false }, back); +} +``` +Keep the existing round-trip tests (Int32/Float/Double/DateTime/Bool) — they assert round-trip, not exact form, so they continue to pass. Keep `Decode_MalformedJson_Throws`. + +**Step 2 — Run, expect FAIL** on the new native-form encode assertions. +Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/ZB.MOM.WW.ScadaBridge.Commons.Tests.csproj --filter AttributeValueCodecTests` + +**Step 3 — Implement.** In `Encode`, replace ONLY the `IEnumerable` branch: + +```csharp +case IEnumerable e: + // Native-typed JSON: serialize the runtime collection so STJ emits numbers/bools + // unquoted, strings quoted, DateTime as ISO-8601. Boxed as object so STJ uses the + // runtime element type. STJ numbers/dates are culture-invariant by spec. + return JsonSerializer.Serialize(e, JsonOpts); +``` +(The `null`, `string`, and `IFormattable` scalar branches are unchanged.) + +In `Decode`, change the list deserialization to read `JsonElement[]` and derive a string per element: + +```csharp +JsonElement[] raw; +try { raw = JsonSerializer.Deserialize(value) ?? []; } +catch (JsonException ex) { throw new FormatException("Malformed list JSON.", ex); } + +var clrType = ElementClrType(elementType.Value); +var listType = typeof(List<>).MakeGenericType(clrType); +var result = (IList)Activator.CreateInstance(listType)!; +foreach (var el in raw) + result.Add(ParseScalar(JsonElementToString(el), elementType.Value)); +return result; +``` +Add the helper (handles both new native elements and old quoted-string elements): + +```csharp +private static string? JsonElementToString(JsonElement el) => el.ValueKind switch +{ + JsonValueKind.String => el.GetString(), // old form, or string-typed lists + JsonValueKind.Null => null, // ParseScalar throws "may not be null" + _ => el.GetRawText() // number/bool → "10" / "1.5" / "true" +}; +``` +`ParseScalar` is unchanged: it parses the string per element type (invariant), so `"10"` (from either form), `"true"`, and ISO/US-invariant dates all parse. Add `using System.Text.Json;` if not already present (it is). + +**Step 4 — Run, expect PASS** (all AttributeValueCodec tests). + +**Step 5 — Commit.** +```bash +git commit -m "feat(commons): native-typed JSON for List values; Decode reads both forms" -- \ + src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs \ + tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs +``` + +**Acceptance:** numbers/bools encode unquoted, strings stay quoted, DateTime is ISO; both old and new forms decode; scalars unchanged; malformed still throws. + +--- + +### Task NJ-2: Stamp ElementDataType on instance-override writes + +**Classification:** small +**Estimated implement time:** ~3 min +**Parallelizable with:** NJ-1 +**Blocked by:** none + +**Files:** +- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Services/InstanceService.cs:146-198` (`SetAttributeOverrideAsync`) +- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Services/InstanceServiceTests.cs` (mirror existing style) + +`SetAttributeOverrideAsync` already loads the base `templateAttr` (line 159). Set `ElementDataType` from it on both the update and create branches so the persisted `InstanceAttributeOverride` row carries the element type — this is what the NJ-3 central normalizer reads for override rows (folds in follow-up #93/M3). + +**Step 1 — Failing test:** setting an override on a `List`/`String` template attribute persists an `InstanceAttributeOverride` whose `ElementDataType == DataType.String`; setting an override on a scalar attribute persists `ElementDataType == null`. + +**Step 2 — Run, expect FAIL.** + +**Step 3 — Implement:** in the update branch, `existingOverride.ElementDataType = templateAttr.ElementDataType;` before the repo update; in the create branch, set `ElementDataType = templateAttr.ElementDataType` in the `new InstanceAttributeOverride(...)` initializer. + +**Step 4 — Run, expect PASS.** + +**Step 5 — Commit:** `feat(template): stamp ElementDataType on instance attribute overrides`. + +--- + +### Task NJ-3: Central MS SQL idempotent startup normalizer + +**Classification:** high-risk +**Estimated implement time:** ~5 min +**Parallelizable with:** NJ-4, NJ-5 (after NJ-1) +**Blocked by:** NJ-1, NJ-2 + +**Files:** +- Create: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ListValueNormalizer.cs` +- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/MigrationHelper.cs` (call the normalizer after the migrate/validate step) +- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/...` (use the existing test DbContext/SQLite-in-memory or test-double pattern the project uses — investigate the existing ConfigurationDatabase tests first) + +**Behavior:** A static `ListValueNormalizer.NormalizeAsync(ScadaBridgeDbContext db, ILogger? logger, CancellationToken ct)`: +- `TemplateAttributes` where `DataType == DataType.List`: for each, compute `var native = AttributeValueCodec.Encode(AttributeValueCodec.Decode(a.Value, DataType.List, a.ElementDataType));` (parse the row's own `ElementDataType` — it is a `DataType?`). If `native != a.Value`, set `a.Value = native`. +- `InstanceAttributeOverrides` whose `ElementDataType != null` (i.e. a List override stamped by NJ-2): same re-encode against `OverrideValue`. Rows with null `ElementDataType` are skipped with a debug log (legacy/scalar rows — none exist in practice since the feature has no deployed data). +- Wrap each row in try/catch(`FormatException`): on a malformed value, log a warning and SKIP that row — continue with the rest. The normalizer MUST NOT throw out of `NormalizeAsync` (never abort startup). +- `SaveChangesAsync` once at the end. Log a summary (`Normalized {n} list attribute value(s) to native JSON`). +- Idempotent: native→native yields identical bytes, so a second run makes no changes. + +**Wire-in (MigrationHelper):** after the `if (isDevelopment) { Migrate } else { validate }` block in `ApplyOrValidateMigrationsAsync`, call `await ListValueNormalizer.NormalizeAsync(dbContext, logger, cancellationToken);`. (Runs on every central startup; idempotent and safe even if both central nodes run it — concurrent UPDATEs set identical values.) + +**Steps (TDD):** failing tests (a TemplateAttributes row in old form `["10","20"]` with ElementDataType Int32 → rewritten to `[10,20]`; a native row → unchanged/no-op; a String-list `["a","b"]` → unchanged; a malformed row → skipped, sibling rows still normalized; an override row with ElementDataType set → normalized, with null → skipped) → run targeted → FAIL → implement `ListValueNormalizer` + wire-in → run targeted → PASS → build ConfigurationDatabase → commit `feat(db): idempotent startup normalizer rewriting List values to native JSON`. + +--- + +### Task NJ-4: Site SQLite active normalization on override load + +**Classification:** high-risk +**Estimated implement time:** ~4 min +**Parallelizable with:** NJ-3, NJ-5 (after NJ-1) +**Blocked by:** NJ-1 + +**Files:** +- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs:932-960` (`HandleOverridesLoaded`) +- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTests.cs` + +In `HandleOverridesLoaded`, the List branch already decodes the stored override via `DecodeAttributeValue(resolved, kvp.Value)`. Extend it: when the decode succeeds and the re-encoded native form differs from the stored string (i.e. the stored value is old-form), re-persist the native form: + +```csharp +var decoded = DecodeAttributeValue(resolved, kvp.Value); +_attributes[kvp.Key] = decoded; +if (decoded is null && !string.IsNullOrEmpty(kvp.Value)) +{ + _attributeQualities[kvp.Key] = "Bad"; +} +else if (decoded is not null) +{ + var native = AttributeValueCodec.Encode(decoded); + if (native != kvp.Value) // stored value was old-form → normalize on disk + { + var key = kvp.Key; + _storage.SetStaticOverrideAsync(_instanceUniqueName, key, native!) + .ContinueWith(t => _logger.LogWarning(t.Exception?.GetBaseException(), + "Failed to normalize static override {Instance}.{Attr} to native JSON", _instanceUniqueName, key), + TaskContinuationOptions.OnlyOnFaulted); + } +} +``` +Idempotent (native→native ⇒ `native == kvp.Value` ⇒ no write). Scalars untouched. Never throws into the actor (decode already FormatException-safe; persist is fire-and-forget). + +**Steps (TDD):** failing tests (an old-form List static override on load → `SetStaticOverrideAsync` called with the native form; a native-form override → NOT re-persisted; a scalar override → unaffected; malformed → Bad quality + no crash, as today) → run targeted `--filter "FullyQualifiedName~InstanceActor"` → FAIL → implement → PASS → build SiteRuntime → commit `feat(siteruntime): normalize old-form List static overrides to native JSON on load`. + +--- + +### Task NJ-5: Normalize List values on bundle import + +**Classification:** standard +**Estimated implement time:** ~4 min +**Parallelizable with:** NJ-3, NJ-4 (after NJ-1) +**Blocked by:** NJ-1 + +**Files:** +- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (the 3 `TemplateAttribute` construction sites from MV-12: BuildTemplate + the two in SyncTemplateAttributesAsync) — and/or `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs` `FromBundleContent` if that is the single choke point for DTO→entity. +- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/...` + +**Behavior:** when importing a `TemplateAttributeDto` whose `DataType == DataType.List`, write the **native** Value: `var v = NormalizeListValue(dto.Value, dto.ElementDataType);` where `NormalizeListValue` does `AttributeValueCodec.Encode(AttributeValueCodec.Decode(value, DataType.List, elementType))` inside try/catch(FormatException) → on failure return the original value unchanged + log (don't fail the import; the MS SQL normalizer is a backstop). Prefer a single private helper used at all construction sites (DRY). Non-List attributes write `dto.Value` unchanged. + +**Steps (TDD):** failing test (import a bundle/DTO with an old-form List attribute `["10","20"]` + ElementDataType Int32 → the persisted TemplateAttribute.Value is `[10,20]`; a String list stays quoted; a malformed value imports unchanged) → run targeted → FAIL → implement the helper + apply at construction sites → PASS → build Transport → commit `feat(transport): normalize List attribute values to native JSON on import`. + +--- + +### Task NJ-6: Integration verification + docs + +**Classification:** standard +**Estimated implement time:** ~5 min +**Parallelizable with:** none +**Blocked by:** NJ-1 … NJ-5 + +**Files:** +- Modify: `docs/plans/2026-06-16-native-typed-json-design.md` (mark Status complete) +- Modify: `docs/requirements/Component-Commons.md` (update the `DataType`/codec note: List values are native-typed JSON; codec reads both forms) +- Modify: `docs/plans/2026-06-16-multivalue-attribute.md.tasks.json` follow-up note OR the task list — note that #93/M3 (override ElementDataType) was folded in here. + +**Steps:** +1. Full-solution build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → 0/0. +2. Targeted test sweep across touched projects: Commons codec, TemplateEngine (InstanceService), ConfigurationDatabase (normalizer), SiteRuntime (InstanceActor), Transport (serializer/importer). +3. Update docs; mark design Status complete. +4. Commit: `docs: mark native-typed JSON feature complete; update Component-Commons codec note`. + +**Acceptance:** full build green; all touched-project targeted tests pass; docs synced. + +--- + +## Parallelization summary + +- **Wave 1:** NJ-1 (codec) ∥ NJ-2 (override stamp) — disjoint (Commons vs TemplateEngine). +- **Wave 2 (after NJ-1, +NJ-2 for NJ-3):** NJ-3 (normalizer), NJ-4 (site), NJ-5 (import) in parallel — disjoint (ConfigurationDatabase vs SiteRuntime vs Transport). +- **Wave 3:** NJ-6 (verify + docs). + +## Risk notes + +- **NJ-1** changes the canonical storage/wire form — high-risk, but backward-compatible (read-both) and additive on the proto field. +- **NJ-3** writes to the config DB at startup — must be idempotent and never abort startup (per-row skip+log). +- **NJ-4** touches the actor model — the normalization write is fire-and-forget and never throws into the actor. +- There is no deployed List data yet, so the normalizers are a safety net; correctness of NJ-1 (the format) is what matters most in practice. diff --git a/docs/plans/2026-06-16-native-typed-json.md.tasks.json b/docs/plans/2026-06-16-native-typed-json.md.tasks.json new file mode 100644 index 00000000..ac42df4f --- /dev/null +++ b/docs/plans/2026-06-16-native-typed-json.md.tasks.json @@ -0,0 +1,16 @@ +{ + "planPath": "docs/plans/2026-06-16-native-typed-json.md", + "designDoc": "docs/plans/2026-06-16-native-typed-json-design.md", + "branch": "feature/native-typed-json", + "status": "complete", + "tasks": [ + {"id": 96, "ref": "NJ-1", "subject": "Codec — native-typed Encode + read-both Decode", "class": "high-risk", "status": "completed", "commits": ["180d554", "bf80ca1"]}, + {"id": 97, "ref": "NJ-2", "subject": "Stamp ElementDataType on instance-override writes (folds in #93/M3)", "class": "small", "status": "completed", "commits": ["abe8832"]}, + {"id": 98, "ref": "NJ-3", "subject": "Central MS SQL idempotent startup normalizer", "class": "high-risk", "status": "completed", "commits": ["f4b101b", "feeae13"]}, + {"id": 99, "ref": "NJ-4", "subject": "Site SQLite active normalization on override load", "class": "high-risk", "status": "completed", "commits": ["5841cec", "feeae13"]}, + {"id": 100, "ref": "NJ-5", "subject": "Normalize List values on bundle import", "class": "standard", "status": "completed", "commits": ["e3d804a", "feeae13"]}, + {"id": 101, "ref": "NJ-6", "subject": "Integration verification + docs", "class": "standard", "status": "completed"} + ], + "lastUpdated": "2026-06-16", + "note": "feature/native-typed-json was reset off these commits during concurrent other-window git work and restored to f4b101b (the NJ-1..NJ-5 tip) via reflog; NJ-3/4/5 review fixes committed as feeae13 + NJ-6 docs on top." +} diff --git a/docs/requirements/Component-Commons.md b/docs/requirements/Component-Commons.md index 87853db4..929b31b7 100644 --- a/docs/requirements/Component-Commons.md +++ b/docs/requirements/Component-Commons.md @@ -26,7 +26,7 @@ Referenced by all component libraries and the Host. Commons must define shared primitive and utility types used across multiple components, including but not limited to: -- **`DataType` enum**: Enumerates the data types supported by the system: Boolean, Int32, Float, Double, String, DateTime, Binary, and **List** (a homogeneous multi-value attribute). A `List` attribute carries a companion `ElementDataType` (one of the scalar types — String, Int32, Float, Double, Boolean, DateTime; not Binary, not nested List) on `TemplateAttribute` / `InstanceAttributeOverride` and the flattened `ResolvedAttribute`. List values are stored and transmitted as a canonical JSON array via the `AttributeValueCodec` helper (scalars keep their historical invariant-culture string form); the `ElementDataType` is fixed by the defining level and cannot be changed on a derived template or instance override. +- **`DataType` enum**: Enumerates the data types supported by the system: Boolean, Int32, Float, Double, String, DateTime, Binary, and **List** (a homogeneous multi-value attribute). A `List` attribute carries a companion `ElementDataType` (one of the scalar types — String, Int32, Float, Double, Boolean, DateTime; not Binary, not nested List) on `TemplateAttribute` / `InstanceAttributeOverride` and the flattened `ResolvedAttribute`. List values are stored and transmitted as a canonical **native-typed** JSON array via the `AttributeValueCodec` helper — numbers and booleans unquoted, strings quoted, `DateTime` as ISO-8601 (e.g. `[10,20]`, `[true,false]`, `["2026-06-16T00:00:00Z"]`); scalars keep their historical invariant-culture string form. `AttributeValueCodec.Decode` is backward compatible: it reads both the native form and the earlier array-of-strings form (`["10","20"]`). Already-persisted old-form values are normalized to native on the fly — an idempotent central startup normalizer (`ListValueNormalizer`, run after migrations) rewrites MS SQL rows, the `InstanceActor` re-persists site SQLite overrides on load, and bundle import re-encodes List values. The `ElementDataType` is fixed by the defining level and cannot be changed on a derived template or instance override. - **`RetryPolicy`**: A record or immutable class describing retry behavior (max retries, fixed delay between retries). - **`Result`**: A discriminated result type that represents either a success value or an error, enabling consistent error handling across component boundaries without exceptions. - **`InstanceState` enum**: Enabled, Disabled. diff --git a/src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs b/src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs index ecc4cbea..b848b79e 100644 --- a/src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs +++ b/src/ZB.MOM.WW.ScadaBridge.Commons/Types/AttributeValueCodec.cs @@ -25,11 +25,10 @@ public static class AttributeValueCodec case string s: return s; // already canonical case IFormattable f: return f.ToString(null, CultureInfo.InvariantCulture); case IEnumerable e: - var items = e.Cast() - .Select(x => x is IFormattable xf - ? xf.ToString(null, CultureInfo.InvariantCulture) - : x?.ToString()); - return JsonSerializer.Serialize(items, JsonOpts); + // Native-typed JSON: serialize the runtime collection so System.Text.Json emits + // numbers/bools unquoted, strings quoted, and DateTime as ISO-8601. Boxed as object + // so STJ uses the runtime element type. STJ numbers/dates are culture-invariant. + return JsonSerializer.Serialize(e, JsonOpts); default: return value.ToString(); } } @@ -46,18 +45,25 @@ public static class AttributeValueCodec if (elementType is null) throw new FormatException("List attribute requires an element type."); - string?[] raw; - try { raw = JsonSerializer.Deserialize(value) ?? []; } + JsonElement[] raw; + try { raw = JsonSerializer.Deserialize(value) ?? []; } catch (JsonException ex) { throw new FormatException("Malformed list JSON.", ex); } var clrType = ElementClrType(elementType.Value); var listType = typeof(List<>).MakeGenericType(clrType); var result = (IList)Activator.CreateInstance(listType)!; - foreach (var item in raw) - result.Add(ParseScalar(item, elementType.Value)); + foreach (var el in raw) + result.Add(ParseScalar(JsonElementToString(el), elementType.Value)); return result; } + private static string? JsonElementToString(JsonElement el) => el.ValueKind switch + { + JsonValueKind.String => el.GetString(), // old form, or string-typed lists + JsonValueKind.Null => null, // ParseScalar throws "may not be null" + _ => el.GetRawText() // number/bool → "10" / "1.5" / "true" + }; + private static Type ElementClrType(DataType t) => t switch { DataType.String => typeof(string), diff --git a/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ListValueNormalizer.cs b/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ListValueNormalizer.cs new file mode 100644 index 00000000..2d8659b6 --- /dev/null +++ b/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ListValueNormalizer.cs @@ -0,0 +1,118 @@ +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.ScadaBridge.Commons.Types; +using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums; + +namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase; + +/// +/// Idempotent central startup normalizer that rewrites already-persisted List attribute +/// values from the old array-of-strings JSON form (["10","20"]) to the new +/// native-typed form ([10,20]). +/// +/// +/// This is a safety net: at the time of writing there is no deployed List attribute data, +/// so in practice it finds nothing to rewrite. It runs on every central startup after +/// migrations apply and MUST never abort startup for data reasons — every row's work is +/// wrapped in a per-row try/catch that logs and skips on malformed data. A second run +/// finds nothing to change (native → native re-encode is byte-identical). +/// +/// +public static class ListValueNormalizer +{ + /// + /// Rewrites old-form List attribute values to the native-typed JSON form across + /// and + /// . Idempotent and + /// best-effort: malformed rows are logged and skipped, never rethrown. + /// + /// The configuration database context. + /// Optional logger for diagnostics. + /// Cancellation token. + /// A task that represents the asynchronous operation. + public static async Task NormalizeAsync( + ScadaBridgeDbContext db, + ILogger? logger = null, + CancellationToken ct = default) + { + var rewritten = 0; + + // TemplateAttributes: List rows carry the element type on the row itself. + var templateRows = await db.TemplateAttributes + .Where(a => a.DataType == DataType.List) + .ToListAsync(ct); + + foreach (var a in templateRows) + { + try + { + var native = AttributeValueCodec.Encode( + AttributeValueCodec.Decode(a.Value, DataType.List, a.ElementDataType)); + if (native != a.Value) + { + a.Value = native; + rewritten++; + } + } + catch (Exception ex) + { + // Never abort startup for a single bad row. FormatException from Decode is the + // expected case; the broad catch also covers an unexpected serialize failure + // (e.g. a JsonException on a non-finite value) so one poison row can't crash boot. + logger?.LogWarning(ex, + "List value normalizer: skipping unprocessable list value for TemplateAttribute {Id}.", + a.Id); + } + } + + // InstanceAttributeOverrides: only rows that carry an element type are List rows. + // Rows with a null ElementDataType are scalar/legacy rows (no deployed List data + // exists, so none in practice) and are skipped. + var overrideRows = await db.InstanceAttributeOverrides + .Where(o => o.ElementDataType != null) + .ToListAsync(ct); + + foreach (var o in overrideRows) + { + try + { + var native = AttributeValueCodec.Encode( + AttributeValueCodec.Decode(o.OverrideValue, DataType.List, o.ElementDataType)); + if (native != o.OverrideValue) + { + o.OverrideValue = native; + rewritten++; + } + } + catch (Exception ex) + { + logger?.LogWarning(ex, + "List value normalizer: skipping unprocessable list value for InstanceAttributeOverride {Id}.", + o.Id); + } + } + + try + { + await db.SaveChangesAsync(ct); + } + catch (Exception ex) + { + // A catastrophic DB failure on SaveChanges may propagate, but log it first so + // startup diagnostics are not silent. Per-row data problems are already handled + // above and never reach here. + logger?.LogError(ex, "List value normalizer: SaveChanges failed."); + throw; + } + + if (rewritten > 0) + { + logger?.LogInformation( + "List value normalizer: rewrote {n} attribute value(s) to native JSON.", rewritten); + } + else + { + logger?.LogDebug("List value normalizer: no attribute values required rewriting."); + } + } +} diff --git a/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/MigrationHelper.cs b/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/MigrationHelper.cs index c5b9ba73..c5b15d1a 100644 --- a/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/MigrationHelper.cs +++ b/src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/MigrationHelper.cs @@ -47,6 +47,12 @@ public static class MigrationHelper "Apply migrations using 'dotnet ef database update' or the generated SQL scripts before starting in production mode."); } } + + // Safety-net normalizer: rewrite any already-persisted List attribute values from the + // old array-of-strings JSON form to the new native-typed form. Idempotent and never + // aborts startup for data reasons (per-row skip + log). Safe even if both central + // nodes run it concurrently on startup. + await ListValueNormalizer.NormalizeAsync(dbContext, logger, cancellationToken); } private static async Task WaitForDatabaseReadyAsync( diff --git a/src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs b/src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs index c8bd3afa..55b4e781 100644 --- a/src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs +++ b/src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/InstanceActor.cs @@ -948,10 +948,33 @@ public class InstanceActor : ReceiveActor if (_resolvedAttributeByName.TryGetValue(kvp.Key, out var resolved) && IsListAttribute(resolved)) { + // NJ-4: decode the stored List override (both old array-of-strings + // and native-typed forms decode) and re-persist the native form if + // the stored value is still in the OLD form. Re-encoding the decoded + // list and comparing to the stored string detects old-form values + // (native → native is byte-identical, so a native value is a no-op). + // The re-persist is fire-and-forget and never throws into the actor. var decoded = DecodeAttributeValue(resolved, kvp.Value); _attributes[kvp.Key] = decoded; if (decoded is null && !string.IsNullOrEmpty(kvp.Value)) + { _attributeQualities[kvp.Key] = "Bad"; + } + else if (decoded is not null) + { + var native = AttributeValueCodec.Encode(decoded); + if (native != kvp.Value) // stored value was old-form → normalize on disk + { + var key = kvp.Key; + var logger = _logger; + var instanceName = _instanceUniqueName; + _storage.SetStaticOverrideAsync(instanceName, key, native!) + .ContinueWith(t => logger.LogWarning(t.Exception?.GetBaseException(), + "Failed to normalize static override {Instance}.{Attr} to native JSON", + instanceName, key), + TaskContinuationOptions.OnlyOnFaulted); + } + } } else { diff --git a/src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Services/InstanceService.cs b/src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Services/InstanceService.cs index 1c69a847..1f859bca 100644 --- a/src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Services/InstanceService.cs +++ b/src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Services/InstanceService.cs @@ -172,6 +172,7 @@ public class InstanceService if (existingOverride != null) { existingOverride.OverrideValue = overrideValue; + existingOverride.ElementDataType = templateAttr.ElementDataType; await _repository.UpdateInstanceAttributeOverrideAsync(existingOverride, cancellationToken); await _repository.SaveChangesAsync(cancellationToken); @@ -185,7 +186,8 @@ public class InstanceService var newOverride = new InstanceAttributeOverride(attributeName) { InstanceId = instanceId, - OverrideValue = overrideValue + OverrideValue = overrideValue, + ElementDataType = templateAttr.ElementDataType }; await _repository.AddInstanceAttributeOverrideAsync(newOverride, cancellationToken); diff --git a/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs b/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs index a93d34bc..43750632 100644 --- a/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs +++ b/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs @@ -1,6 +1,7 @@ using System.IO.Compression; using System.Security.Cryptography; using System.Text.Json; +using Microsoft.Extensions.Logging; using Microsoft.Extensions.Options; using ZB.MOM.WW.ScadaBridge.Commons.Entities.ExternalSystems; using ZB.MOM.WW.ScadaBridge.Commons.Entities.InboundApi; @@ -70,6 +71,7 @@ public sealed class BundleImporter : IBundleImporter private readonly IOptions _options; private readonly TimeProvider _timeProvider; private readonly SemanticValidator _semanticValidator; + private readonly ILogger? _logger; /// /// Initializes a new with all required dependencies. @@ -106,7 +108,8 @@ public sealed class BundleImporter : IBundleImporter IAuditService auditService, IAuditCorrelationContext correlationContext, ScadaBridgeDbContext dbContext, - SemanticValidator semanticValidator) + SemanticValidator semanticValidator, + ILogger? logger = null) { _bundleSerializer = bundleSerializer ?? throw new ArgumentNullException(nameof(bundleSerializer)); _manifestValidator = manifestValidator ?? throw new ArgumentNullException(nameof(manifestValidator)); @@ -124,6 +127,7 @@ public sealed class BundleImporter : IBundleImporter _correlationContext = correlationContext ?? throw new ArgumentNullException(nameof(correlationContext)); _dbContext = dbContext ?? throw new ArgumentNullException(nameof(dbContext)); _semanticValidator = semanticValidator ?? throw new ArgumentNullException(nameof(semanticValidator)); + _logger = logger; } /// @@ -1040,7 +1044,7 @@ public sealed class BundleImporter : IBundleImporter { t.Attributes.Add(new TemplateAttribute(a.Name) { - Value = a.Value, + Value = ImportValueNormalizer.NormalizeListValue(a.Value, a.DataType, a.ElementDataType), DataType = a.DataType, IsLocked = a.IsLocked, Description = a.Description, @@ -1115,11 +1119,17 @@ public sealed class BundleImporter : IBundleImporter // Adds + Updates. foreach (var attrDto in dto.Attributes) { + // Normalise List values to the native-typed JSON form on import so the + // comparison (and the persisted value) match what the target already + // stores natively — otherwise an idempotent re-import of an old-form + // bundle would spuriously report a Value change. + var normalizedValue = ImportValueNormalizer.NormalizeListValue( + attrDto.Value, attrDto.DataType, attrDto.ElementDataType, _logger, attrDto.Name); if (existingByName.TryGetValue(attrDto.Name, out var current)) { // Update only if any field actually changed. bool changed = - !string.Equals(current.Value, attrDto.Value, StringComparison.Ordinal) || + !string.Equals(current.Value, normalizedValue, StringComparison.Ordinal) || current.DataType != attrDto.DataType || current.IsLocked != attrDto.IsLocked || !string.Equals(current.Description, attrDto.Description, StringComparison.Ordinal) || @@ -1127,7 +1137,7 @@ public sealed class BundleImporter : IBundleImporter current.ElementDataType != attrDto.ElementDataType; if (!changed) continue; - current.Value = attrDto.Value; + current.Value = normalizedValue; current.DataType = attrDto.DataType; current.IsLocked = attrDto.IsLocked; current.Description = attrDto.Description; @@ -1157,7 +1167,7 @@ public sealed class BundleImporter : IBundleImporter { var newAttr = new TemplateAttribute(attrDto.Name) { - Value = attrDto.Value, + Value = normalizedValue, DataType = attrDto.DataType, IsLocked = attrDto.IsLocked, Description = attrDto.Description, diff --git a/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs b/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs index 22c7c31b..88b66573 100644 --- a/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs +++ b/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs @@ -199,7 +199,7 @@ public sealed class EntitySerializer t.Attributes.Add(new TemplateAttribute(a.Name) { TemplateId = t.Id, - Value = a.Value, + Value = ImportValueNormalizer.NormalizeListValue(a.Value, a.DataType, a.ElementDataType), DataType = a.DataType, IsLocked = a.IsLocked, Description = a.Description, diff --git a/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/ImportValueNormalizer.cs b/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/ImportValueNormalizer.cs new file mode 100644 index 00000000..2fcf9772 --- /dev/null +++ b/src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/ImportValueNormalizer.cs @@ -0,0 +1,63 @@ +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.ScadaBridge.Commons.Types; +using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums; + +namespace ZB.MOM.WW.ScadaBridge.Transport.Serialization; + +/// +/// Import-time normalization of attribute values to the native-typed JSON form. +/// +/// Bundles exported before the native-typed-JSON change carry List attribute +/// values in the old quoted-element form (e.g. ["10","20"] for an +/// list). Already-exported bundle files can't be +/// rewritten, so List values are normalised on import: every DTO→entity write +/// site routes the value through so imported +/// data lands native ([10,20]). The central DB normalizer remains the +/// backstop for anything that slips through. +/// +/// +/// Non-List attributes and null/empty values pass through unchanged. A value +/// that fails to decode (malformed JSON / un-parseable element) is left exactly +/// as-is so the import still succeeds — the DB normalizer is the backstop. +/// +/// +internal static class ImportValueNormalizer +{ + /// + /// Returns the native-typed JSON form of a List attribute value, or the + /// value unchanged for non-List / null / empty / malformed inputs. + /// + /// The attribute value as carried by the bundle DTO. + /// The attribute's declared data type. + /// The List element type (null for scalars). + /// Optional logger; a warning is emitted when a malformed value is left as-is. + /// Optional attribute name for the diagnostic message. + public static string? NormalizeListValue( + string? value, + DataType dataType, + DataType? elementType, + ILogger? logger = null, + string? attributeName = null) + { + if (dataType != DataType.List || string.IsNullOrEmpty(value)) + { + return value; + } + + try + { + return AttributeValueCodec.Encode( + AttributeValueCodec.Decode(value, DataType.List, elementType)); + } + catch (FormatException ex) + { + // Leave malformed values exactly as imported; the DB normalizer is + // the backstop. Never abort the import for a single bad value. + logger?.LogWarning(ex, + "Bundle import: could not normalize List value for attribute {Attribute}; " + + "importing verbatim (the central DB normalizer is the backstop).", + attributeName ?? "(unknown)"); + return value; + } + } +} diff --git a/tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs b/tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs index a4aed551..fa717bdb 100644 --- a/tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs +++ b/tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Types/AttributeValueCodecTests.cs @@ -64,7 +64,7 @@ public class AttributeValueCodecTests try { CultureInfo.CurrentCulture = new CultureInfo("de-DE"); - Assert.Equal("[\"1.5\",\"2.5\"]", + Assert.Equal("[1.5,2.5]", AttributeValueCodec.Encode(new List { 1.5, 2.5 })); } finally @@ -73,6 +73,71 @@ public class AttributeValueCodecTests } } + [Fact] + public void Encode_Int32List_ProducesNativeNumbers() => + Assert.Equal("[10,20,30]", + AttributeValueCodec.Encode(new List { 10, 20, 30 })); + + [Fact] + public void Encode_BoolList_ProducesNativeBooleans() => + Assert.Equal("[true,false]", + AttributeValueCodec.Encode(new List { true, false })); + + [Fact] + public void Encode_StringList_StaysQuoted() => + Assert.Equal("[\"a\",\"b\"]", + AttributeValueCodec.Encode(new List { "a", "b" })); + + [Fact] + public void Encode_DateTimeList_IsIso8601() + { + var json = AttributeValueCodec.Encode( + new List { new(2026, 6, 16, 0, 0, 0, DateTimeKind.Utc) }); + Assert.StartsWith("[\"", json); // DateTime list must be an array of quoted strings + Assert.Contains("2026-06-16T00:00:00", json); + Assert.DoesNotContain("06/16/2026", json); + } + + [Fact] + public void Decode_OldStringFloatForm_BackwardCompatible() + { + var back = (IList)AttributeValueCodec.Decode("[\"1.5\",\"2.25\"]", DataType.List, DataType.Float)!; + Assert.Equal(new[] { 1.5f, 2.25f }, back); + } + + [Fact] + public void Decode_OldStringDateTimeForm_BackwardCompatible() + { + // The pre-native codec encoded DateTime via IFormattable.ToString(null, InvariantCulture): + // "06/16/2026 12:30:45" (US-invariant, no 'T'/'Z'). New Decode must still parse it. + var back = (IList)AttributeValueCodec.Decode( + "[\"06/16/2026 12:30:45\"]", DataType.List, DataType.DateTime)!; + Assert.Equal(new DateTime(2026, 6, 16, 12, 30, 45, DateTimeKind.Unspecified), back[0]); + } + + [Fact] + public void Decode_NewNativeIntForm_Parses() + { + var back = (IList)AttributeValueCodec.Decode("[10,20]", DataType.List, DataType.Int32)!; + Assert.Equal(new[] { 10, 20 }, back); + } + + [Fact] + public void Decode_OldStringIntForm_BackwardCompatible() + { + var back = (IList)AttributeValueCodec.Decode("[\"10\",\"20\"]", DataType.List, DataType.Int32)!; + Assert.Equal(new[] { 10, 20 }, back); + } + + [Theory] + [InlineData("[true,false]")] + [InlineData("[\"True\",\"False\"]")] + public void Decode_BoolForms_BothParse(string json) + { + var back = (IList)AttributeValueCodec.Decode(json, DataType.List, DataType.Boolean)!; + Assert.Equal(new[] { true, false }, back); + } + [Fact] public void RoundTrip_Int32List() { diff --git a/tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/ListValueNormalizerTests.cs b/tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/ListValueNormalizerTests.cs new file mode 100644 index 00000000..fc224c46 --- /dev/null +++ b/tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/ListValueNormalizerTests.cs @@ -0,0 +1,231 @@ +using Microsoft.EntityFrameworkCore; +using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances; +using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites; +using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates; +using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums; +using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase; + +namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests; + +/// +/// Tests for — the idempotent startup normalizer that +/// rewrites already-persisted List attribute values from the old array-of-strings JSON +/// form (["10","20"]) to the new native-typed form ([10,20]). +/// +public class ListValueNormalizerTests : IDisposable +{ + private readonly ScadaBridgeDbContext _context; + + public ListValueNormalizerTests() + { + _context = SqliteTestHelper.CreateInMemoryContext(); + } + + public void Dispose() + { + _context.Database.CloseConnection(); + _context.Dispose(); + } + + // Seeds a Template parent (satisfies the TemplateAttribute -> Template FK) and returns its Id. + private async Task SeedTemplateAsync() + { + var template = new Template("T1"); + _context.Templates.Add(template); + await _context.SaveChangesAsync(); + return template.Id; + } + + // Seeds Site + Template + Instance (satisfies the InstanceAttributeOverride -> Instance FK) + // and returns the Instance Id. + private async Task SeedInstanceAsync() + { + var site = new Site("Site1", "S-001"); + var template = new Template("T1"); + _context.Sites.Add(site); + _context.Templates.Add(template); + await _context.SaveChangesAsync(); + + var instance = new Instance("Inst1") { SiteId = site.Id, TemplateId = template.Id }; + _context.Instances.Add(instance); + await _context.SaveChangesAsync(); + return instance.Id; + } + + [Fact] + public async Task TemplateAttribute_OldStringForm_RewrittenToNative() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("intList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.Int32, + Value = "[\"10\",\"20\"]", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + var reloaded = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("[10,20]", reloaded.Value); + } + + [Fact] + public async Task TemplateAttribute_AlreadyNative_IsNotRewritten() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("intList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.Int32, + Value = "[10,20]", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + // No tracked entity should be marked Modified — idempotent no-op. + var tracked = _context.ChangeTracker.Entries() + .Where(e => e.State == EntityState.Modified) + .ToList(); + Assert.Empty(tracked); + + var reloaded = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("[10,20]", reloaded.Value); + } + + [Fact] + public async Task TemplateAttribute_StringList_IsUnchanged() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("stringList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.String, + Value = "[\"a\",\"b\"]", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + var reloaded = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("[\"a\",\"b\"]", reloaded.Value); + } + + [Fact] + public async Task TemplateAttribute_Malformed_IsSkipped_AndSiblingStillNormalized() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("badList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.Int32, + Value = "[\"a\"", // malformed JSON + }); + _context.TemplateAttributes.Add(new TemplateAttribute("goodList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.Int32, + Value = "[\"5\"]", // old form, valid + }); + await _context.SaveChangesAsync(); + + // Must NOT throw despite the malformed row. + await ListValueNormalizer.NormalizeAsync(_context); + + var bad = await _context.TemplateAttributes.AsNoTracking() + .SingleAsync(a => a.Name == "badList"); + var good = await _context.TemplateAttributes.AsNoTracking() + .SingleAsync(a => a.Name == "goodList"); + + Assert.Equal("[\"a\"", bad.Value); // skipped, untouched + Assert.Equal("[5]", good.Value); // normalized + } + + [Fact] + public async Task TemplateAttribute_NonListRow_IsUnchanged() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("scalar") + { + TemplateId = templateId, + DataType = DataType.Int32, + ElementDataType = null, + Value = "42", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + var reloaded = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("42", reloaded.Value); + } + + [Fact] + public async Task InstanceAttributeOverride_OldStringForm_RewrittenToNative() + { + var instanceId = await SeedInstanceAsync(); + _context.InstanceAttributeOverrides.Add(new InstanceAttributeOverride("intList") + { + InstanceId = instanceId, + ElementDataType = DataType.Int32, + OverrideValue = "[\"5\"]", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + var reloaded = await _context.InstanceAttributeOverrides.AsNoTracking().SingleAsync(); + Assert.Equal("[5]", reloaded.OverrideValue); + } + + [Fact] + public async Task InstanceAttributeOverride_NullElementType_IsUntouched() + { + var instanceId = await SeedInstanceAsync(); + _context.InstanceAttributeOverrides.Add(new InstanceAttributeOverride("scalar") + { + InstanceId = instanceId, + ElementDataType = null, + OverrideValue = "42", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + + var reloaded = await _context.InstanceAttributeOverrides.AsNoTracking().SingleAsync(); + Assert.Equal("42", reloaded.OverrideValue); + } + + [Fact] + public async Task NormalizeAsync_IsIdempotent_SecondRunChangesNothing() + { + var templateId = await SeedTemplateAsync(); + _context.TemplateAttributes.Add(new TemplateAttribute("intList") + { + TemplateId = templateId, + DataType = DataType.List, + ElementDataType = DataType.Int32, + Value = "[\"10\",\"20\"]", + }); + await _context.SaveChangesAsync(); + + await ListValueNormalizer.NormalizeAsync(_context); + var afterFirst = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("[10,20]", afterFirst.Value); + + await ListValueNormalizer.NormalizeAsync(_context); + var afterSecond = await _context.TemplateAttributes.AsNoTracking().SingleAsync(); + Assert.Equal("[10,20]", afterSecond.Value); + + var tracked = _context.ChangeTracker.Entries() + .Where(e => e.State == EntityState.Modified) + .ToList(); + Assert.Empty(tracked); + } +} diff --git a/tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTests.cs b/tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTests.cs index f93a238e..72dbb49c 100644 --- a/tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTests.cs +++ b/tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Actors/InstanceActorTests.cs @@ -1016,4 +1016,159 @@ public class InstanceActorTests : TestKit, IDisposable Assert.Single(overrides); Assert.Equal("[]", overrides["Labels"]); } + + // ── NJ-4: old-form List static override normalization on load ──────────── + + /// + /// NJ-4: an OLD array-of-strings static override (["10","20"]) for an + /// Int32 List attribute must be re-persisted in the native form ([10,20]) + /// when the actor loads it at startup. The in-memory read still returns the + /// typed list {10,20}; the on-disk value is normalized to native JSON. + /// + [Fact] + public async Task InstanceActor_OldFormListOverride_NormalizedToNativeOnLoad() + { + await _storage.SetStaticOverrideAsync("Pump-OldForm", "Counts", "[\"10\",\"20\"]"); + + var config = new FlattenedConfiguration + { + InstanceUniqueName = "Pump-OldForm", + Attributes = + [ + new ResolvedAttribute + { + CanonicalName = "Counts", Value = "[1,2]", + DataType = "List", ElementDataType = "Int32" + } + ] + }; + + var actor = CreateInstanceActor("Pump-OldForm", config); + + // Wait for the async override load (PipeTo) + fire-and-forget normalization. + await Task.Delay(1000); + + // In-memory read returns the typed list, decoded from the old form. + actor.Tell(new GetAttributeRequest("corr-of", "Pump-OldForm", "Counts", DateTimeOffset.UtcNow)); + var response = ExpectMsg(TimeSpan.FromSeconds(5)); + Assert.True(response.Found); + var list = Assert.IsType>(response.Value); + Assert.Equal(new[] { 10, 20 }, list); + + // The on-disk override has been normalized to the native form. + var overrides = await _storage.GetStaticOverridesAsync("Pump-OldForm"); + Assert.Single(overrides); + Assert.Equal("[10,20]", overrides["Counts"]); + } + + /// + /// NJ-4: a NATIVE-form static override ([10,20]) is already canonical, so + /// load-time normalization must be a no-op — the on-disk value is unchanged + /// (idempotent: native → native is byte-identical, so no re-persist occurs). + /// + [Fact] + public async Task InstanceActor_NativeFormListOverride_NotRePersistedOnLoad() + { + await _storage.SetStaticOverrideAsync("Pump-Native", "Counts", "[10,20]"); + + var config = new FlattenedConfiguration + { + InstanceUniqueName = "Pump-Native", + Attributes = + [ + new ResolvedAttribute + { + CanonicalName = "Counts", Value = "[1,2]", + DataType = "List", ElementDataType = "Int32" + } + ] + }; + + var actor = CreateInstanceActor("Pump-Native", config); + await Task.Delay(1000); + + actor.Tell(new GetAttributeRequest("corr-nat", "Pump-Native", "Counts", DateTimeOffset.UtcNow)); + var response = ExpectMsg(TimeSpan.FromSeconds(5)); + Assert.True(response.Found); + var list = Assert.IsType>(response.Value); + Assert.Equal(new[] { 10, 20 }, list); + + // The native value is left untouched on disk. + var overrides = await _storage.GetStaticOverridesAsync("Pump-Native"); + Assert.Single(overrides); + Assert.Equal("[10,20]", overrides["Counts"]); + } + + /// + /// NJ-4: a scalar static override is unaffected by the List normalization path — + /// its on-disk value is left exactly as stored (no native re-encode). + /// + [Fact] + public async Task InstanceActor_ScalarOverride_NotTouchedByListNormalization() + { + await _storage.SetStaticOverrideAsync("Pump-ScalarOf", "Temperature", "200.0"); + + var config = new FlattenedConfiguration + { + InstanceUniqueName = "Pump-ScalarOf", + Attributes = + [ + new ResolvedAttribute { CanonicalName = "Temperature", Value = "100.0", DataType = "Double" } + ] + }; + + var actor = CreateInstanceActor("Pump-ScalarOf", config); + await Task.Delay(1000); + + actor.Tell(new GetAttributeRequest("corr-sof", "Pump-ScalarOf", "Temperature", DateTimeOffset.UtcNow)); + var response = ExpectMsg(TimeSpan.FromSeconds(5)); + Assert.True(response.Found); + Assert.Equal("200.0", response.Value); + + var overrides = await _storage.GetStaticOverridesAsync("Pump-ScalarOf"); + Assert.Single(overrides); + Assert.Equal("200.0", overrides["Temperature"]); + } + + /// + /// NJ-4: a malformed stored List override (truncated JSON) must NOT crash the + /// actor and must NOT be re-persisted — it loads with Bad quality (as today), + /// the actor stays alive, and the poison on-disk value is left unchanged. + /// + [Fact] + public async Task InstanceActor_MalformedListOverride_BadQuality_NotRePersisted() + { + await _storage.SetStaticOverrideAsync("Pump-BadOf", "Counts", "[\"a\""); + + var config = new FlattenedConfiguration + { + InstanceUniqueName = "Pump-BadOf", + Attributes = + [ + new ResolvedAttribute + { + CanonicalName = "Counts", Value = "[1,2]", + DataType = "List", ElementDataType = "Int32" + } + ] + }; + + var actor = CreateInstanceActor("Pump-BadOf", config); + Watch(actor); + await Task.Delay(1000); + + actor.Tell(new GetAttributeRequest("corr-bof", "Pump-BadOf", "Counts", DateTimeOffset.UtcNow)); + var response = ExpectMsg(TimeSpan.FromSeconds(5)); + Assert.True(response.Found); + Assert.Equal("Bad", response.Quality); + Assert.Null(response.Value); + + // The actor must still be alive — no crash from the normalization path. + ExpectNoTerminated(actor, TimeSpan.FromMilliseconds(500)); + + // The malformed value must NOT have been re-persisted (left exactly as stored). + var overrides = await _storage.GetStaticOverridesAsync("Pump-BadOf"); + Assert.Single(overrides); + Assert.Equal("[\"a\"", overrides["Counts"]); + } } diff --git a/tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Services/InstanceServiceTests.cs b/tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Services/InstanceServiceTests.cs index b1cbc66b..b7e0a7fc 100644 --- a/tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Services/InstanceServiceTests.cs +++ b/tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Services/InstanceServiceTests.cs @@ -269,6 +269,106 @@ public class InstanceServiceTests _repoMock.Verify(r => r.AddInstanceConnectionBindingAsync(It.IsAny(), It.IsAny()), Times.Exactly(2)); } + // --- NJ-2: ElementDataType stamping on SetAttributeOverrideAsync --- + + [Fact] + public async Task SetAttributeOverride_CreatePath_ListAttribute_StampsElementDataType() + { + var instance = new Instance("Inst1") { Id = 1, TemplateId = 1 }; + _repoMock.Setup(r => r.GetInstanceByIdAsync(1, It.IsAny())) + .ReturnsAsync(instance); + _repoMock.Setup(r => r.GetAttributesByTemplateIdAsync(1, It.IsAny())) + .ReturnsAsync(new List + { + new("Tags") + { + IsLocked = false, + DataType = DataType.List, + ElementDataType = DataType.String + } + }); + _repoMock.Setup(r => r.GetOverridesByInstanceIdAsync(1, It.IsAny())) + .ReturnsAsync(new List()); // no existing override → create path + _repoMock.Setup(r => r.SaveChangesAsync(It.IsAny())) + .ReturnsAsync(1); + + InstanceAttributeOverride? captured = null; + _repoMock.Setup(r => r.AddInstanceAttributeOverrideAsync(It.IsAny(), It.IsAny())) + .Callback((o, _) => captured = o); + + var result = await _sut.SetAttributeOverrideAsync(1, "Tags", "[\"a\"]", "admin"); + + Assert.True(result.IsSuccess); + Assert.NotNull(captured); + Assert.Equal(DataType.String, captured.ElementDataType); + } + + [Fact] + public async Task SetAttributeOverride_UpdatePath_ListAttribute_StampsElementDataType() + { + var instance = new Instance("Inst1") { Id = 1, TemplateId = 1 }; + _repoMock.Setup(r => r.GetInstanceByIdAsync(1, It.IsAny())) + .ReturnsAsync(instance); + _repoMock.Setup(r => r.GetAttributesByTemplateIdAsync(1, It.IsAny())) + .ReturnsAsync(new List + { + new("Tags") + { + IsLocked = false, + DataType = DataType.List, + ElementDataType = DataType.String + } + }); + + var existingOverride = new InstanceAttributeOverride("Tags") { Id = 42, InstanceId = 1, OverrideValue = "[\"old\"]" }; + _repoMock.Setup(r => r.GetOverridesByInstanceIdAsync(1, It.IsAny())) + .ReturnsAsync(new List { existingOverride }); // pre-existing → update path + _repoMock.Setup(r => r.SaveChangesAsync(It.IsAny())) + .ReturnsAsync(1); + + InstanceAttributeOverride? captured = null; + _repoMock.Setup(r => r.UpdateInstanceAttributeOverrideAsync(It.IsAny(), It.IsAny())) + .Callback((o, _) => captured = o); + + var result = await _sut.SetAttributeOverrideAsync(1, "Tags", "[\"new\"]", "admin"); + + Assert.True(result.IsSuccess); + Assert.NotNull(captured); + Assert.Equal(DataType.String, captured.ElementDataType); + } + + [Fact] + public async Task SetAttributeOverride_CreatePath_ScalarAttribute_ElementDataTypeIsNull() + { + var instance = new Instance("Inst1") { Id = 1, TemplateId = 1 }; + _repoMock.Setup(r => r.GetInstanceByIdAsync(1, It.IsAny())) + .ReturnsAsync(instance); + _repoMock.Setup(r => r.GetAttributesByTemplateIdAsync(1, It.IsAny())) + .ReturnsAsync(new List + { + new("Threshold") + { + IsLocked = false, + DataType = DataType.Float, + ElementDataType = null + } + }); + _repoMock.Setup(r => r.GetOverridesByInstanceIdAsync(1, It.IsAny())) + .ReturnsAsync(new List()); + _repoMock.Setup(r => r.SaveChangesAsync(It.IsAny())) + .ReturnsAsync(1); + + InstanceAttributeOverride? captured = null; + _repoMock.Setup(r => r.AddInstanceAttributeOverrideAsync(It.IsAny(), It.IsAny())) + .Callback((o, _) => captured = o); + + var result = await _sut.SetAttributeOverrideAsync(1, "Threshold", "3.14", "admin"); + + Assert.True(result.IsSuccess); + Assert.NotNull(captured); + Assert.Null(captured.ElementDataType); + } + [Fact] public async Task AssignToArea_AreaInDifferentSite_ReturnsFailure() { diff --git a/tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Serialization/EntitySerializerTests.cs b/tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Serialization/EntitySerializerTests.cs index b4fefbb0..51f8ab0c 100644 --- a/tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Serialization/EntitySerializerTests.cs +++ b/tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Serialization/EntitySerializerTests.cs @@ -259,6 +259,122 @@ public sealed class EntitySerializerTests Assert.Equal("[\"a\",\"b\"]", rtAttr.Value); } + private static BundleContentDto MakeContentWithListAttribute( + string value, DataType elementType) + { + var template = new TemplateDto( + Name: "Pump", + FolderName: null, + BaseTemplateName: null, + Description: null, + Attributes: new[] + { + new TemplateAttributeDto( + Name: "Tags", + Value: value, + DataType: DataType.List, + IsLocked: false, + Description: null, + DataSourceReference: null, + ElementDataType: elementType), + }, + Alarms: Array.Empty(), + Scripts: Array.Empty(), + Compositions: Array.Empty()); + + return new BundleContentDto( + TemplateFolders: Array.Empty(), + Templates: new[] { template }, + SharedScripts: Array.Empty(), + ExternalSystems: Array.Empty(), + DatabaseConnections: Array.Empty(), + NotificationLists: Array.Empty(), + SmtpConfigs: Array.Empty(), + ApiMethods: Array.Empty()); + } + + [Fact] + public void Import_normalizes_old_form_Int32_list_value_to_native_json() + { + // Pre-native bundle: quoted Int32 list elements. + var dto = MakeContentWithListAttribute("[\"10\",\"20\"]", DataType.Int32); + + var aggregate = new EntitySerializer().FromBundleContent(dto); + + var attr = Assert.Single(Assert.Single(aggregate.Templates).Attributes); + Assert.Equal(DataType.List, attr.DataType); + Assert.Equal(DataType.Int32, attr.ElementDataType); + // Imported native: numbers unquoted. + Assert.Equal("[10,20]", attr.Value); + } + + [Fact] + public void Import_leaves_string_list_value_quoted() + { + var dto = MakeContentWithListAttribute("[\"a\",\"b\"]", DataType.String); + + var aggregate = new EntitySerializer().FromBundleContent(dto); + + var attr = Assert.Single(Assert.Single(aggregate.Templates).Attributes); + Assert.Equal(DataType.List, attr.DataType); + Assert.Equal(DataType.String, attr.ElementDataType); + // Strings stay quoted in native form. + Assert.Equal("[\"a\",\"b\"]", attr.Value); + } + + [Fact] + public void Import_leaves_malformed_list_value_unchanged_without_throwing() + { + // Truncated JSON array — Decode throws FormatException; the import must + // still succeed and carry the value through verbatim (DB normalizer is + // the backstop). + var dto = MakeContentWithListAttribute("[\"a\"", DataType.String); + + var aggregate = new EntitySerializer().FromBundleContent(dto); + + var attr = Assert.Single(Assert.Single(aggregate.Templates).Attributes); + Assert.Equal("[\"a\"", attr.Value); + } + + [Fact] + public void Import_leaves_scalar_attribute_value_unchanged() + { + var template = new TemplateDto( + Name: "Sensor", + FolderName: null, + BaseTemplateName: null, + Description: null, + Attributes: new[] + { + new TemplateAttributeDto( + Name: "Pressure", + Value: "42.0", + DataType: DataType.Double, + IsLocked: false, + Description: null, + DataSourceReference: null, + ElementDataType: null), + }, + Alarms: Array.Empty(), + Scripts: Array.Empty(), + Compositions: Array.Empty()); + var dto = new BundleContentDto( + TemplateFolders: Array.Empty(), + Templates: new[] { template }, + SharedScripts: Array.Empty(), + ExternalSystems: Array.Empty(), + DatabaseConnections: Array.Empty(), + NotificationLists: Array.Empty(), + SmtpConfigs: Array.Empty(), + ApiMethods: Array.Empty()); + + var aggregate = new EntitySerializer().FromBundleContent(dto); + + var attr = Assert.Single(Assert.Single(aggregate.Templates).Attributes); + Assert.Equal(DataType.Double, attr.DataType); + Assert.Equal("42.0", attr.Value); + } + [Fact] public void Roundtrip_scalar_attribute_with_null_ElementDataType_remains_null() {