feat(historian): advertise only the backend's aggregates under AggregateFunctions (#682)
IHistorianDataSource.SupportedAggregates (default: every HistoryAggregateType) is the one authority for both ServerCapabilities/AggregateFunctions (i=2997) and HistoryServerCapabilities/AggregateFunctions (i=11201) and for the up-front BadAggregateNotSupported in HistoryRead-Processed. Gateway declares AggregateModeMapper.Served (4), SqlServer/QuestDB AggregateFamilies.Served (17), Null none. Only the folder references are trimmed; the SDK's calculator factories stay so live AggregateFilter subscriptions keep working. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
@@ -1353,6 +1353,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (10, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted seven (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM` — plus the **duration-based** PercentGood/PercentBad, #680) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`, percentages included). ⚠️ **#680 status rules (client-visible):** TimeAverage/Total are Uncertain if any part of the interval uses an Uncertain value, is bridged across a Bad sample, is extrapolated past the last sample (⇒ a live tag's latest interval is Uncertain) or is not covered; TimeAverage2/Total2 keep the SDK time-based rule (Uncertain time counts as Good) but never extrapolate past the last sample — that interval is Partial, later ones `BadNoData` (a report-by-exception tag with nothing after its last sample reads `BadNoData` until its next sample arrives). `docs/Historian.md` §"Time-weighted aggregates". ⚠️ **#680 per-sample (client-visible):** `Count` counts **Good** samples only (value-based status; an all-Bad interval is generic Bad, value null) and an empty interval **inside the tag's data range** is `0`/Good (outside it still `BadNoData`); `Start`/`End` return the **raw** first/last sample of any quality — its own timestamp and status, value null when Bad; `Delta` is Uncertain only when the interval's first or last sample is unusable; and every per-sample aggregate but `Average` sets **end-of-data Partial** on the interval holding the last sample (forward incl. its earlier edge, reverse strictly inside), never on a Bad result. The data-range lookup is a seek on SQL Server but a partition scan on QuestDB (unindexed `tag` SYMBOL). `docs/Historian.md` §"Aggregates". Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention.
|
||||
- **QuestDB** (`Driver.Historian.QuestDb`, `ServerHistorian:QuestDb:*`, Npgsql over PG wire; docker-dev overlay `docker-compose.questdb.yml`): same kinds/status/aggregates/HistoryUpdate as SqlServer, soft delete by tombstone (INSERT only — never `UPDATE`/`DELETE`; every read filters `NOT deleted`). ⚠️ **Timestamps are µs** (`QuestDbConnection.Round`/`BindTimestamp`; `IHistorianDataSource.TimestampResolutionTicks` = 10 steps HistoryRead paging), **pre-1970 cannot be stored** (writer drops, HistoryUpdate `BadOutOfRange`), every write waits for WAL visibility, TTL is enforced on write commits only, set Npgsql `Maximum Pool Size` < QuestDB's 64. `docs/Historian.md` §"QuestDB backend".
|
||||
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
|
||||
- **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `AggregateModeMapper.Served`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**) is the one authority for both `AggregateFunctions` folders (i=2997, i=11201 — only the folder references are trimmed; the SDK's calculator factories stay for live `AggregateFilter` subscriptions) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates.
|
||||
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
||||
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
|
||||
- **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization AND after planning — an unservable request (Remove / `IsDeleteModified` / not historized) stays `BadHistoryOperationUnsupported` off-Primary; the verdict is `IHistoryUpdateRoleView` = the data-plane verdict but **closed until this node's first redundancy snapshot**, so a pair cannot both accept in the boot window; admin-only nodes register no view ⇒ no gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ node-level `BadHistoryOperationUnsupported` on every arm (the updater throws `NotSupportedException`, nothing sent; the `HistoryWrite` bit is still advertised — not knowable at materialise time), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key).
|
||||
@@ -1362,7 +1363,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
|
||||
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
|
||||
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
|
||||
(historized nodes return `GoodNoData`, never an error). Supported variants: Raw, Processed
|
||||
(historized nodes return `GoodNoData` for Raw/AtTime — Processed is `BadAggregateNotSupported` since #682). Supported variants: Raw, Processed
|
||||
(backend-dependent: Gateway serves TimeAverage/Minimum/Maximum/Total, with Average/Count `BadAggregateNotSupported` since AVEVA has no mean or sample count, #679; SqlServer/QuestDb serve seventeen), and AtTime over historized variable nodes; Events over
|
||||
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
|
||||
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
|
||||
|
||||
+42
-8
@@ -95,7 +95,9 @@ HistoryRead is served from any node (the database is shared). HistoryUpdate is *
|
||||
|
||||
The `ServerHistorian` section in `appsettings.json` controls the historian read path.
|
||||
`Enabled` defaults to `false`; when disabled, the server registers `NullHistorianDataSource`
|
||||
and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error).
|
||||
and Raw / AtTime / Events HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error); a
|
||||
Processed read is `BadAggregateNotSupported` and no aggregate functions are advertised
|
||||
([#682](#advertised-aggregates-aggregatefunctions-682)).
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -563,7 +565,7 @@ node without `isHistorized` has no HistoryRead access bit and answers
|
||||
|---|---|
|
||||
| Historized node, historian configured and reachable, results found | `Good` |
|
||||
| Historized node, historian configured, time range empty | `GoodNoData` |
|
||||
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) |
|
||||
| Historized node, historian NOT configured (`Enabled=false` / Null source) | `GoodNoData` (empty) for Raw / AtTime; Processed ⇒ `BadAggregateNotSupported` (#682) |
|
||||
| Non-historized node | `BadHistoryOperationUnsupported` |
|
||||
| Backend timeout or exception | `BadHistoryOperationUnsupported` per node; other nodes in the same batch are unaffected |
|
||||
| `Backend=SqlServer`, `AutoCreateSchema=false`, schema missing/behind | every read/write fails with a clear error naming the missing schema version (health `LastError`); reads answer `BadHistoryOperationUnsupported` per node; no host crash |
|
||||
@@ -764,14 +766,46 @@ VPN. Reverse processed reads on the `Gateway` backend are answered by AVEVA and
|
||||
|
||||
`HistoryAggregateType` carries the wider Part 13 set (Interpolative, TimeAverage, Start, End, Delta, Range,
|
||||
Min/MaxActualTime, Start/EndBound, DeltaBounds, DurationGood/Bad, PercentGood/Bad, WorstQuality,
|
||||
NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopulation); `OtOpcUaNodeManager.MapAggregate`
|
||||
maps each standard aggregate NodeId (`TimeAverage2` and `Total2` were appended for #671). The **`SqlServer`** and
|
||||
NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopulation); `AggregateFunctionMap`
|
||||
(`OpcUaServer`) maps each standard aggregate NodeId (`TimeAverage2` and `Total2` were appended for #671). The **`SqlServer`** and
|
||||
**`QuestDb`** backends serve the seventeen listed in the [aggregate table](#aggregates-processed-reads); everything else
|
||||
(e.g. `StartBound`, `DurationGood`, `StandardDeviationPopulation`) is `BadAggregateNotSupported` for that node. The **HistorianGateway** backend only has retrieval modes for the four
|
||||
(TimeAverage, Minimum, Maximum, Total): for any other member `AggregateModeMapper` throws
|
||||
`HistoryAggregateNotSupportedException`, which the HistoryRead path turns into `BadAggregateNotSupported` for that
|
||||
node only (not a generic failure, and not counted as a backend fault). The OPC UA Client driver forwards every
|
||||
member upstream. A NodeId outside the mapped set (e.g. `StandardDeviationSample`) is still `BadAggregateNotSupported`.
|
||||
(TimeAverage, Minimum, Maximum, Total). A NodeId outside the mapped set (e.g. `StandardDeviationSample`) is still
|
||||
`BadAggregateNotSupported`.
|
||||
|
||||
### Advertised aggregates (`AggregateFunctions`, #682)
|
||||
|
||||
Each backend declares the aggregates it serves as `IHistorianDataSource.SupportedAggregates` — **the single authority** for
|
||||
two things:
|
||||
|
||||
- **What the server advertises.** `Server/ServerCapabilities/AggregateFunctions` (`i=2997`) and
|
||||
`HistoryServerCapabilities/AggregateFunctions` (`i=11201`) — two separate folder nodes that organize the same
|
||||
aggregate-function objects — list exactly that set. Before #682 both listed the SDK's full 37 regardless of backend.
|
||||
- **What HistoryRead-Processed refuses up front.** An aggregate outside the set is `BadAggregateNotSupported` for that node
|
||||
without a backend call (the backend still throws `HistoryAggregateNotSupportedException` for one as defence in depth —
|
||||
derived from the same table, never a second list).
|
||||
|
||||
| `ServerHistorian:Backend` | Advertised | Declared by |
|
||||
|---|---|---|
|
||||
| `Gateway` | 4 — TimeAverage, Minimum, Maximum, Total | `AggregateModeMapper.Served` (derived from its mapping switch) |
|
||||
| `SqlServer`, `QuestDb` | 17 — the [aggregate table](#aggregates-processed-reads) | `AggregateFamilies.Served` (Sql.Common; every member with a family) |
|
||||
| historian disabled (Null source) | **none** | `NullHistorianDataSource` |
|
||||
|
||||
⚠️ **Client-visible: with no historian configured a processed HistoryRead is now `BadAggregateNotSupported`, not
|
||||
`GoodNoData`.** An aggregate is a statistic the server cannot compute without a history, and the empty folders now say so;
|
||||
Raw / AtTime / Events reads on the Null source still return the empty `GoodNoData` result.
|
||||
|
||||
How it is wired: `OtOpcUaSdkServer` keeps the SDK's `CreateAggregateManager` (all 37 calculator factories) and then trims
|
||||
only the folder ⇄ aggregate `Organizes` references to the node manager's current source — the Null default at boot; the
|
||||
Host's `SetHistorianDataSource` (the backend resolved once from `ServerHistorian:Backend`) re-syncs it right after start,
|
||||
and every later assignment re-syncs both directions. ⚠️ **The calculator factories are deliberately NOT removed**: a
|
||||
live-data MonitoredItem with an `AggregateFilter` is validated and computed by the SDK's `AggregateManager`, not the
|
||||
historian, so live aggregate subscriptions keep working for every SDK aggregate, advertised or not (pinned over the wire
|
||||
by `AdvertisedAggregateFunctionsWireTests`). The aggregate-function object nodes themselves stay readable. A boot log line
|
||||
names the advertised set.
|
||||
|
||||
Not changed: the `HistoryServerCapabilities` booleans (`AccessHistoryDataCapability`, `InsertDataCapability`, …) are still
|
||||
whatever the SDK sets — #682 touches only the two folders (follow-up candidate).
|
||||
|
||||
### Known limitations
|
||||
|
||||
|
||||
@@ -55,6 +55,18 @@ public enum HistoryAggregateType
|
||||
Total2,
|
||||
}
|
||||
|
||||
/// <summary>Shared <see cref="HistoryAggregateType"/> sets for <see cref="IHistorianDataSource.SupportedAggregates"/> (#682).</summary>
|
||||
public static class HistoryAggregateTypes
|
||||
{
|
||||
/// <summary>Every defined <see cref="HistoryAggregateType"/> — the interface default.</summary>
|
||||
public static readonly IReadOnlySet<HistoryAggregateType> All =
|
||||
System.Collections.Frozen.FrozenSet.ToFrozenSet(Enum.GetValues<HistoryAggregateType>());
|
||||
|
||||
/// <summary>No aggregates — a source with no history to aggregate (the Null historian).</summary>
|
||||
public static readonly IReadOnlySet<HistoryAggregateType> None =
|
||||
System.Collections.Frozen.FrozenSet<HistoryAggregateType>.Empty;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Thrown by a history backend when it cannot serve the requested <see cref="HistoryAggregateType"/>.
|
||||
/// The Server HistoryRead path translates it to <c>BadAggregateNotSupported</c> for that node only.
|
||||
|
||||
@@ -174,6 +174,23 @@ public interface IHistorianDataSource : IDisposable
|
||||
/// </remarks>
|
||||
long TimestampResolutionTicks => 1;
|
||||
|
||||
/// <summary>
|
||||
/// The Part 13 aggregates this backend serves for a processed HistoryRead (Gitea #682) — the ONE authority for
|
||||
/// both what the server advertises under <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201), and what its HistoryRead-Processed answers
|
||||
/// <c>BadAggregateNotSupported</c> up front without calling the backend.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The default is <see cref="HistoryAggregateTypes.All"/> — "every aggregate the server can name; a member I cannot
|
||||
/// serve throws <see cref="HistoryAggregateNotSupportedException"/> at read time", which is exactly the pre-#682
|
||||
/// behaviour, so a test fake or an out-of-tree source keeps working unchanged. It is a concrete set rather than a
|
||||
/// <c>null</c> "all" sentinel so every consumer handles one shape. Every PRODUCTION backend declares its set itself
|
||||
/// (a reflection guard pins that): a wrapper that forgot to forward this would otherwise advertise every aggregate.
|
||||
/// The set is read once when the source is wired (the backend is fixed at boot by <c>ServerHistorian:Backend</c>)
|
||||
/// and again on every processed read, so it must be cheap and stable — return a static frozen set.
|
||||
/// </remarks>
|
||||
IReadOnlySet<HistoryAggregateType> SupportedAggregates => HistoryAggregateTypes.All;
|
||||
|
||||
/// <summary>
|
||||
/// Read modified-value history — OPC UA HistoryReadRaw with <c>IsReadModified = true</c> (Part 11, Gitea #668):
|
||||
/// every modification a HistoryUpdate applied to the tag inside the window, each carrying the PRIOR value.
|
||||
|
||||
@@ -74,6 +74,15 @@ public sealed class NullHistorianDataSource : IHistorianDataSource
|
||||
? throw new NotSupportedException("The null historian cannot read events newest-first (reverse).")
|
||||
: Task.FromResult(EmptyEvents);
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
/// EMPTY (Gitea #682): with no historian configured the server advertises no aggregate functions, and a processed
|
||||
/// HistoryRead answers <c>BadAggregateNotSupported</c> per node rather than <c>GoodNoData</c> — an aggregate is a
|
||||
/// statistic the server cannot compute without a history, and advertising the SDK's full list on a server with no
|
||||
/// historian misled capability discovery. Raw / AtTime / Events reads still return the empty <c>GoodNoData</c> result.
|
||||
/// </remarks>
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => HistoryAggregateTypes.None;
|
||||
|
||||
/// <inheritdoc />
|
||||
public HistorianHealthSnapshot GetHealthSnapshot() => new(
|
||||
TotalQueries: 0,
|
||||
|
||||
@@ -132,6 +132,13 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
||||
/// </remarks>
|
||||
public bool SupportsReverseEvents => false;
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
/// <see cref="AggregateModeMapper.Served"/> — the aggregates the gateway has a retrieval mode for (#682); every
|
||||
/// other member answers <c>BadAggregateNotSupported</c> and is not advertised.
|
||||
/// </remarks>
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => AggregateModeMapper.Served;
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
/// A reverse (newest-first) read is refused with <see cref="NotSupportedException"/> — the server reports
|
||||
|
||||
+24
-7
@@ -1,3 +1,4 @@
|
||||
using System.Collections.Frozen;
|
||||
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
@@ -22,22 +23,38 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Mapping;
|
||||
/// </remarks>
|
||||
internal static class AggregateModeMapper
|
||||
{
|
||||
/// <summary>
|
||||
/// The aggregates the gateway serves — every defined member <see cref="TryMap"/> maps, derived from it so the
|
||||
/// advertised set and the dispatch can never disagree. <see cref="GatewayHistorianDataSource.SupportedAggregates"/>
|
||||
/// returns it (#682); it is the ONLY declaration of the gateway's served set.
|
||||
/// </summary>
|
||||
public static readonly IReadOnlySet<HistoryAggregateType> Served =
|
||||
Enum.GetValues<HistoryAggregateType>().Where(a => TryMap(a) is not null).ToFrozenSet();
|
||||
|
||||
/// <summary>Maps an aggregate function to the gateway retrieval mode.</summary>
|
||||
/// <param name="aggregate">The driver-agnostic aggregate function.</param>
|
||||
/// <returns>The matching gateway <see cref="RetrievalMode"/>.</returns>
|
||||
/// <exception cref="HistoryAggregateNotSupportedException">A defined member the gateway has no retrieval mode for.</exception>
|
||||
/// <exception cref="ArgumentOutOfRangeException">An undefined enum value.</exception>
|
||||
public static RetrievalMode ToRetrievalMode(HistoryAggregateType aggregate) => aggregate switch
|
||||
public static RetrievalMode ToRetrievalMode(HistoryAggregateType aggregate)
|
||||
{
|
||||
if (TryMap(aggregate) is { } mode) return mode;
|
||||
|
||||
// Part 13 aggregates the gateway has no retrieval mode for (#285) — an UNDEFINED enum value is a
|
||||
// programming error, but a defined-yet-unserved member is a normal client request the server
|
||||
// answers with BadAggregateNotSupported.
|
||||
if (Enum.IsDefined(aggregate)) throw new HistoryAggregateNotSupportedException(aggregate);
|
||||
throw new ArgumentOutOfRangeException(
|
||||
nameof(aggregate), aggregate, "Unmapped HistoryAggregateType — add a RetrievalMode mapping.");
|
||||
}
|
||||
|
||||
/// <summary>The single mapping table: the retrieval mode for a served aggregate, or null.</summary>
|
||||
private static RetrievalMode? TryMap(HistoryAggregateType aggregate) => aggregate switch
|
||||
{
|
||||
HistoryAggregateType.TimeAverage => RetrievalMode.TimeWeightedAverage,
|
||||
HistoryAggregateType.Minimum => RetrievalMode.MinimumWithTime,
|
||||
HistoryAggregateType.Maximum => RetrievalMode.MaximumWithTime,
|
||||
HistoryAggregateType.Total => RetrievalMode.Integral,
|
||||
// Part 13 aggregates the gateway has no retrieval mode for (#285) — an UNDEFINED enum value is a
|
||||
// programming error, but a defined-yet-unserved member is a normal client request the server
|
||||
// answers with BadAggregateNotSupported.
|
||||
_ when Enum.IsDefined(aggregate) => throw new HistoryAggregateNotSupportedException(aggregate),
|
||||
_ => throw new ArgumentOutOfRangeException(
|
||||
nameof(aggregate), aggregate, "Unmapped HistoryAggregateType — add a RetrievalMode mapping."),
|
||||
_ => null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -359,6 +359,10 @@ public sealed class QuestDbHistorianDataSource : IHistorianDataSource
|
||||
/// <remarks>Reverse events reads are served natively (<c>ORDER BY … DESC</c>, Gitea #675).</remarks>
|
||||
public bool SupportsReverseEvents => true;
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>The seventeen <see cref="AggregateFamilies.Served"/> — shared with the other SQL backend (#682).</remarks>
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => AggregateFamilies.Served;
|
||||
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using System.Collections.Frozen;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
@@ -40,4 +41,12 @@ public static class AggregateFamilies
|
||||
or HistoryAggregateType.PercentGood or HistoryAggregateType.PercentBad => AggregateFamily.TimeWeighted,
|
||||
_ => null,
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// The seventeen aggregates the SQL backends serve — every member <see cref="FamilyOf"/> assigns a family, derived
|
||||
/// from it so the set and the dispatch can never disagree. Both <c>SqlServerHistorianDataSource</c> and
|
||||
/// <c>QuestDbHistorianDataSource</c> return it as <see cref="IHistorianDataSource.SupportedAggregates"/> (#682).
|
||||
/// </summary>
|
||||
public static readonly IReadOnlySet<HistoryAggregateType> Served =
|
||||
Enum.GetValues<HistoryAggregateType>().Where(a => FamilyOf(a) is not null).ToFrozenSet();
|
||||
}
|
||||
|
||||
+4
@@ -316,6 +316,10 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
||||
/// <remarks>Reverse events reads are served natively (<c>ORDER BY … DESC</c>, Gitea #675).</remarks>
|
||||
public bool SupportsReverseEvents => true;
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>The seventeen <see cref="AggregateFamilies.Served"/> — shared with the other SQL backend (#682).</remarks>
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => AggregateFamilies.Served;
|
||||
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
using System.Reflection;
|
||||
using Opc.Ua;
|
||||
using Opc.Ua.Server;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — make the two advertised aggregate-function folders, <c>ServerCapabilities/AggregateFunctions</c>
|
||||
/// (i=2997) and <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201), list exactly the aggregates the wired
|
||||
/// historian backend serves (<see cref="IHistorianDataSource.SupportedAggregates"/>), instead of the SDK's full
|
||||
/// registered set.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// The SDK's <c>AggregateManager.RegisterFactory</c> both registers a calculator factory AND adds an
|
||||
/// <c>Organizes</c> reference from each folder to the aggregate's object node. Only the REFERENCES are trimmed
|
||||
/// here: the factories stay, because a live-data MonitoredItem with an <c>AggregateFilter</c> is validated
|
||||
/// against (and computed by) the AggregateManager, not the historian — removing a factory would break live
|
||||
/// aggregate subscriptions for every aggregate the historian happens not to serve.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The two folders are separate nodes that reference the same aggregate object nodes (both directions). The
|
||||
/// aggregate nodes themselves are kept (they are standard type-system nodes a client may still Read); only the
|
||||
/// folder ⇄ aggregate <c>Organizes</c> pair is removed or restored. Idempotent and reversible, so re-wiring a
|
||||
/// different source re-syncs both folders.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class AdvertisedAggregateFunctions
|
||||
{
|
||||
/// <summary>Every standard aggregate-function NodeId the SDK's nodeset knows (<c>ObjectIds.AggregateFunction_*</c>).</summary>
|
||||
private static readonly NodeId[] StandardAggregateIds = typeof(ObjectIds)
|
||||
.GetFields(BindingFlags.Public | BindingFlags.Static)
|
||||
.Where(f => f.FieldType == typeof(NodeId) && f.Name.StartsWith("AggregateFunction_", StringComparison.Ordinal))
|
||||
.Select(f => (NodeId)f.GetValue(null)!)
|
||||
.ToArray();
|
||||
|
||||
/// <summary>The two advertised folders (both are trimmed — they are separate nodes).</summary>
|
||||
internal static readonly NodeId[] Folders =
|
||||
[
|
||||
ObjectIds.Server_ServerCapabilities_AggregateFunctions,
|
||||
ObjectIds.HistoryServerCapabilities_AggregateFunctions,
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// Sync both folders to <paramref name="served"/>: an aggregate-function node is organized under a folder iff the
|
||||
/// server maps it (<see cref="AggregateFunctionMap"/>) AND <paramref name="served"/> contains it.
|
||||
/// </summary>
|
||||
/// <param name="server">The running server.</param>
|
||||
/// <param name="served">The backend's served aggregates.</param>
|
||||
/// <returns>The advertised NodeIds, or <c>null</c> when the diagnostics node manager is not available yet.</returns>
|
||||
public static IReadOnlyList<NodeId>? Apply(IServerInternal server, IReadOnlySet<HistoryAggregateType> served)
|
||||
{
|
||||
if (server?.DiagnosticsNodeManager is not CustomNodeManager2 diagnostics) return null;
|
||||
|
||||
var advertised = new List<NodeId>();
|
||||
lock (diagnostics.Lock)
|
||||
{
|
||||
var folders = Folders
|
||||
.Select(id => diagnostics.FindPredefinedNode(id, typeof(BaseObjectState)))
|
||||
.Where(f => f is not null)
|
||||
.ToList();
|
||||
|
||||
foreach (var aggregateId in StandardAggregateIds)
|
||||
{
|
||||
var aggregateNode = diagnostics.FindPredefinedNode(aggregateId, null);
|
||||
if (aggregateNode is null) continue;
|
||||
|
||||
var advertise = AggregateFunctionMap.Map(aggregateId) is { } aggregate && served.Contains(aggregate);
|
||||
if (advertise) advertised.Add(aggregateId);
|
||||
|
||||
foreach (var folder in folders)
|
||||
{
|
||||
var linked = folder!.ReferenceExists(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
if (advertise && !linked)
|
||||
{
|
||||
folder.AddReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.AddReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
else if (!advertise && linked)
|
||||
{
|
||||
folder.RemoveReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.RemoveReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return advertised;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
using System.Collections.Frozen;
|
||||
using Opc.Ua;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>
|
||||
/// The one table between OPC UA Part 13 standard aggregate-function NodeIds (<c>ObjectIds.AggregateFunction_*</c>)
|
||||
/// and <see cref="HistoryAggregateType"/>. HistoryRead-Processed maps a request's aggregate through it, and the
|
||||
/// advertised <c>AggregateFunctions</c> folders (#682) map a backend's served set back through it — one table, so a
|
||||
/// NodeId the server can dispatch is exactly a NodeId it can advertise.
|
||||
/// </summary>
|
||||
internal static class AggregateFunctionMap
|
||||
{
|
||||
/// <summary>NodeId ⇒ aggregate, for every aggregate the server can name to a backend.</summary>
|
||||
public static readonly FrozenDictionary<NodeId, HistoryAggregateType> ByNodeId = new Dictionary<NodeId, HistoryAggregateType>
|
||||
{
|
||||
[ObjectIds.AggregateFunction_Average] = HistoryAggregateType.Average,
|
||||
[ObjectIds.AggregateFunction_Minimum] = HistoryAggregateType.Minimum,
|
||||
[ObjectIds.AggregateFunction_Maximum] = HistoryAggregateType.Maximum,
|
||||
[ObjectIds.AggregateFunction_Total] = HistoryAggregateType.Total,
|
||||
[ObjectIds.AggregateFunction_Count] = HistoryAggregateType.Count,
|
||||
[ObjectIds.AggregateFunction_Interpolative] = HistoryAggregateType.Interpolative,
|
||||
[ObjectIds.AggregateFunction_TimeAverage] = HistoryAggregateType.TimeAverage,
|
||||
[ObjectIds.AggregateFunction_Start] = HistoryAggregateType.Start,
|
||||
[ObjectIds.AggregateFunction_End] = HistoryAggregateType.End,
|
||||
[ObjectIds.AggregateFunction_Delta] = HistoryAggregateType.Delta,
|
||||
[ObjectIds.AggregateFunction_Range] = HistoryAggregateType.Range,
|
||||
[ObjectIds.AggregateFunction_MinimumActualTime] = HistoryAggregateType.MinimumActualTime,
|
||||
[ObjectIds.AggregateFunction_MaximumActualTime] = HistoryAggregateType.MaximumActualTime,
|
||||
[ObjectIds.AggregateFunction_StartBound] = HistoryAggregateType.StartBound,
|
||||
[ObjectIds.AggregateFunction_EndBound] = HistoryAggregateType.EndBound,
|
||||
[ObjectIds.AggregateFunction_DeltaBounds] = HistoryAggregateType.DeltaBounds,
|
||||
[ObjectIds.AggregateFunction_DurationGood] = HistoryAggregateType.DurationGood,
|
||||
[ObjectIds.AggregateFunction_DurationBad] = HistoryAggregateType.DurationBad,
|
||||
[ObjectIds.AggregateFunction_PercentGood] = HistoryAggregateType.PercentGood,
|
||||
[ObjectIds.AggregateFunction_PercentBad] = HistoryAggregateType.PercentBad,
|
||||
[ObjectIds.AggregateFunction_WorstQuality] = HistoryAggregateType.WorstQuality,
|
||||
[ObjectIds.AggregateFunction_NumberOfTransitions] = HistoryAggregateType.NumberOfTransitions,
|
||||
[ObjectIds.AggregateFunction_AnnotationCount] = HistoryAggregateType.AnnotationCount,
|
||||
[ObjectIds.AggregateFunction_StandardDeviationPopulation] = HistoryAggregateType.StandardDeviationPopulation,
|
||||
[ObjectIds.AggregateFunction_VariancePopulation] = HistoryAggregateType.VariancePopulation,
|
||||
[ObjectIds.AggregateFunction_TimeAverage2] = HistoryAggregateType.TimeAverage2,
|
||||
[ObjectIds.AggregateFunction_Total2] = HistoryAggregateType.Total2,
|
||||
}.ToFrozenDictionary();
|
||||
|
||||
/// <summary>
|
||||
/// Map a request's aggregate-function NodeId to <see cref="HistoryAggregateType"/>; <c>null</c> for one the server
|
||||
/// cannot name to any backend (the caller answers <c>BadAggregateNotSupported</c>).
|
||||
/// </summary>
|
||||
/// <param name="aggregateNodeId">The per-node aggregate-function NodeId from the request.</param>
|
||||
/// <returns>The mapped aggregate, or <c>null</c>.</returns>
|
||||
public static HistoryAggregateType? Map(NodeId? aggregateNodeId) =>
|
||||
aggregateNodeId is not null && ByNodeId.TryGetValue(aggregateNodeId, out var aggregate) ? aggregate : null;
|
||||
}
|
||||
@@ -344,11 +344,48 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
/// <see cref="NodeWriteGateway"/>, the HistoryRead override does NOT run under the
|
||||
/// node-manager <c>Lock</c>, so the override may block-bridge to this (async) source.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Gitea #682: assigning a source also re-syncs the advertised <c>AggregateFunctions</c> folders to its
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/> (see <see cref="SyncAdvertisedAggregates"/>).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public IHistorianDataSource HistorianDataSource
|
||||
{
|
||||
get => _historianDataSource;
|
||||
set => _historianDataSource = value ?? NullHistorianDataSource.Instance;
|
||||
set
|
||||
{
|
||||
_historianDataSource = value ?? NullHistorianDataSource.Instance;
|
||||
SyncAdvertisedAggregates();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — trim <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) to the current source's
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/>. Called on every <see cref="HistorianDataSource"/>
|
||||
/// assignment and by <see cref="OtOpcUaSdkServer"/> right after the SDK registers its aggregate factories, so the
|
||||
/// Null default is applied from boot. Only the folder references change — the AggregateManager's calculator
|
||||
/// factories (live <c>AggregateFilter</c> subscriptions) are untouched. Never throws: a failure leaves the previous
|
||||
/// advertisement in place and is logged, because HistoryRead's own up-front check still enforces the set.
|
||||
/// </summary>
|
||||
internal void SyncAdvertisedAggregates()
|
||||
{
|
||||
try
|
||||
{
|
||||
var source = _historianDataSource;
|
||||
var advertised = AdvertisedAggregateFunctions.Apply(Server, source.SupportedAggregates);
|
||||
if (advertised is not null)
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"OtOpcUaNodeManager: advertising {Count} aggregate function(s) for historian {Source}: {Aggregates}",
|
||||
advertised.Count, source.GetType().Name,
|
||||
string.Join(", ", advertised.Select(id => AggregateFunctionMap.Map(id)?.ToString() ?? id.ToString())));
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "OtOpcUaNodeManager: failed to sync the advertised aggregate functions");
|
||||
}
|
||||
}
|
||||
|
||||
private volatile IHistorianDataUpdater? _historianDataUpdater;
|
||||
@@ -5054,6 +5091,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// #677: a client AggregateConfiguration other than the server defaults is refused per node, never silently ignored.
|
||||
var configurationServable = ServerAggregateConfiguration.IsServable(details.AggregateConfiguration);
|
||||
nodesToProcess = AuthorizeHistoryRead(context, nodesToProcess, results, errors);
|
||||
var source = HistorianDataSource;
|
||||
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||
{
|
||||
using var deadline = CreateDeadline();
|
||||
@@ -5064,8 +5102,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// AggregateType is a per-node parallel collection (same length as nodesToRead, enforced by
|
||||
// the base dispatcher). handle.Index is the node's position in that collection.
|
||||
var aggregateNodeId = details.AggregateType[handle.Index];
|
||||
var aggregate = MapAggregate(aggregateNodeId);
|
||||
if (aggregate is null)
|
||||
// #682: one authority — the source's SupportedAggregates (also what the AggregateFunctions folders
|
||||
// advertise). An aggregate the server cannot name, or the backend does not serve, is refused up front
|
||||
// without a backend call.
|
||||
var aggregate = AggregateFunctionMap.Map(aggregateNodeId);
|
||||
if (aggregate is null || !source.SupportedAggregates.Contains(aggregate.Value))
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadAggregateNotSupported;
|
||||
continue;
|
||||
@@ -6478,45 +6519,6 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
return new HistoryData { DataValues = values };
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Map an OPC UA Part 13 standard-aggregate function NodeId to our
|
||||
/// <see cref="HistoryAggregateType"/>. Returns <c>null</c> for any aggregate we don't serve so
|
||||
/// the caller can surface <c>BadAggregateNotSupported</c>.
|
||||
/// </summary>
|
||||
/// <param name="aggregateNodeId">The per-node aggregate-function NodeId from the request.</param>
|
||||
/// <returns>The mapped aggregate, or <c>null</c> when unsupported.</returns>
|
||||
private static HistoryAggregateType? MapAggregate(NodeId aggregateNodeId)
|
||||
{
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Average) return HistoryAggregateType.Average;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Minimum) return HistoryAggregateType.Minimum;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Maximum) return HistoryAggregateType.Maximum;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Total) return HistoryAggregateType.Total;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Count) return HistoryAggregateType.Count;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Interpolative) return HistoryAggregateType.Interpolative;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_TimeAverage) return HistoryAggregateType.TimeAverage;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Start) return HistoryAggregateType.Start;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_End) return HistoryAggregateType.End;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Delta) return HistoryAggregateType.Delta;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Range) return HistoryAggregateType.Range;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_MinimumActualTime) return HistoryAggregateType.MinimumActualTime;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_MaximumActualTime) return HistoryAggregateType.MaximumActualTime;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_StartBound) return HistoryAggregateType.StartBound;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_EndBound) return HistoryAggregateType.EndBound;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_DeltaBounds) return HistoryAggregateType.DeltaBounds;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_DurationGood) return HistoryAggregateType.DurationGood;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_DurationBad) return HistoryAggregateType.DurationBad;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_PercentGood) return HistoryAggregateType.PercentGood;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_PercentBad) return HistoryAggregateType.PercentBad;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_WorstQuality) return HistoryAggregateType.WorstQuality;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_NumberOfTransitions) return HistoryAggregateType.NumberOfTransitions;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_AnnotationCount) return HistoryAggregateType.AnnotationCount;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_StandardDeviationPopulation) return HistoryAggregateType.StandardDeviationPopulation;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_VariancePopulation) return HistoryAggregateType.VariancePopulation;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_TimeAverage2) return HistoryAggregateType.TimeAverage2;
|
||||
if (aggregateNodeId == ObjectIds.AggregateFunction_Total2) return HistoryAggregateType.Total2;
|
||||
return null;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Project the historian source's <see cref="HistorianRead"/> (Core.Abstractions DTO) into an
|
||||
/// SDK <see cref="HistoryData"/> — one <see cref="DataValue"/> per <see cref="DataValueSnapshot"/>,
|
||||
|
||||
@@ -200,6 +200,21 @@ public sealed class OtOpcUaSdkServer : StandardServer
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — keep the SDK's full set of calculator factories (live-data <c>AggregateFilter</c> subscriptions
|
||||
/// need them), then trim the advertised <c>AggregateFunctions</c> folders to the node manager's current historian
|
||||
/// source. At this point that is the Null default (no aggregates); the Host's
|
||||
/// <see cref="SetHistorianDataSource"/> re-syncs to the configured backend's set right after start.
|
||||
/// </summary>
|
||||
/// <inheritdoc />
|
||||
protected override AggregateManager CreateAggregateManager(
|
||||
IServerInternal server, ApplicationConfiguration configuration)
|
||||
{
|
||||
var manager = base.CreateAggregateManager(server, configuration);
|
||||
_otOpcUaNodeManager?.SyncAdvertisedAggregates();
|
||||
return manager;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override MasterNodeManager CreateMasterNodeManager(
|
||||
IServerInternal server, ApplicationConfiguration configuration)
|
||||
|
||||
+15
@@ -63,4 +63,19 @@ public sealed class HistorianDataSourceReverseEventsGuardTests
|
||||
$"{type.FullName} must declare SupportsReverseEvents itself, not inherit the interface default");
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Every_implementation_declares_its_supported_aggregates_itself()
|
||||
{
|
||||
// Gitea #682: the interface default is "every aggregate" — a production backend (or a future wrapper) that
|
||||
// inherited it would advertise all of them under AggregateFunctions and forward reads it cannot serve.
|
||||
foreach (var type in ProductionImplementations())
|
||||
{
|
||||
var map = type.GetInterfaceMap(typeof(IHistorianDataSource));
|
||||
var getter = map.TargetMethods[Array.FindIndex(map.InterfaceMethods,
|
||||
m => m.Name == "get_" + nameof(IHistorianDataSource.SupportedAggregates))];
|
||||
getter.DeclaringType.ShouldBe(type,
|
||||
$"{type.FullName} must declare SupportedAggregates itself, not inherit the interface default");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+15
@@ -38,6 +38,21 @@ public sealed class AggregateModeMapperTests
|
||||
public void Average_and_Count_are_not_supported(HistoryAggregateType a)
|
||||
=> Assert.Throws<HistoryAggregateNotSupportedException>(() => AggregateModeMapper.ToRetrievalMode(a));
|
||||
|
||||
[Fact] // #682: the served (= advertised) set is exactly what the mapper maps, and the data source returns it
|
||||
public void Served_is_exactly_the_mapped_members_and_the_data_source_declares_it()
|
||||
{
|
||||
Assert.Equal(
|
||||
new[] { HistoryAggregateType.Minimum, HistoryAggregateType.Maximum, HistoryAggregateType.Total, HistoryAggregateType.TimeAverage }
|
||||
.OrderBy(a => a),
|
||||
AggregateModeMapper.Served.OrderBy(a => a));
|
||||
foreach (var a in Enum.GetValues<HistoryAggregateType>().Where(a => !AggregateModeMapper.Served.Contains(a)))
|
||||
Assert.Throws<HistoryAggregateNotSupportedException>(() => AggregateModeMapper.ToRetrievalMode(a));
|
||||
|
||||
var source = new GatewayHistorianDataSource(
|
||||
new FakeHistorianGatewayClient(), Microsoft.Extensions.Logging.Abstractions.NullLogger<GatewayHistorianDataSource>.Instance);
|
||||
Assert.Same(AggregateModeMapper.Served, source.SupportedAggregates);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Undefined_value_is_a_programming_error_not_a_not_supported()
|
||||
=> Assert.Throws<ArgumentOutOfRangeException>(() => AggregateModeMapper.ToRetrievalMode((HistoryAggregateType)9999));
|
||||
|
||||
+9
@@ -19,4 +19,13 @@ public sealed class ProcessedReadTests
|
||||
HistoryAggregateType.PercentGood, HistoryAggregateType.PercentBad, HistoryAggregateType.TimeAverage2,
|
||||
HistoryAggregateType.Total2]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Served_is_exactly_the_members_with_a_family()
|
||||
{
|
||||
// #682: the advertised set and the dispatch are one definition — a member is served iff it has a family.
|
||||
AggregateFamilies.Served.Count.ShouldBe(17);
|
||||
foreach (var a in Enum.GetValues<HistoryAggregateType>())
|
||||
AggregateFamilies.Served.Contains(a).ShouldBe(AggregateFamilies.FamilyOf(a) is not null, a.ToString());
|
||||
}
|
||||
}
|
||||
|
||||
+29
@@ -129,4 +129,33 @@ public sealed class HistorianBackendRegistryTests
|
||||
o.ConnectionString.ShouldContain("Port=18812");
|
||||
o.TablePrefix.ShouldBe("rig_");
|
||||
}
|
||||
|
||||
/// <summary>The seventeen aggregates the SQL backends serve, spelled out (not derived) so a drift in either the
|
||||
/// family table or a backend's declaration fails here (#682).</summary>
|
||||
private static readonly HistoryAggregateType[] SqlServed =
|
||||
[
|
||||
HistoryAggregateType.Average, HistoryAggregateType.Minimum, HistoryAggregateType.Maximum, HistoryAggregateType.Count,
|
||||
HistoryAggregateType.Range, HistoryAggregateType.Start, HistoryAggregateType.End, HistoryAggregateType.Delta,
|
||||
HistoryAggregateType.MinimumActualTime, HistoryAggregateType.MaximumActualTime, HistoryAggregateType.TimeAverage,
|
||||
HistoryAggregateType.TimeAverage2, HistoryAggregateType.Total, HistoryAggregateType.Total2,
|
||||
HistoryAggregateType.Interpolative, HistoryAggregateType.PercentGood, HistoryAggregateType.PercentBad,
|
||||
];
|
||||
|
||||
[Fact]
|
||||
public void Each_backend_data_source_declares_the_aggregates_it_serves()
|
||||
{
|
||||
// #682: what the server advertises under AggregateFunctions comes from these — resolved exactly as the Host does.
|
||||
var sp = new ServiceCollection().BuildServiceProvider();
|
||||
IHistorianDataSource Source(ServerHistorianOptions o, IConfiguration c) =>
|
||||
HistorianBackendRegistry.Resolve(o, c).CreateDataSource(sp);
|
||||
|
||||
Source(Gateway(), Cfg).SupportedAggregates.ShouldBe(
|
||||
[HistoryAggregateType.TimeAverage, HistoryAggregateType.Minimum, HistoryAggregateType.Maximum, HistoryAggregateType.Total],
|
||||
ignoreOrder: true);
|
||||
Source(new ServerHistorianOptions { Backend = "SqlServer", Enabled = true }, SqlCfg())
|
||||
.SupportedAggregates.ShouldBe(SqlServed, ignoreOrder: true);
|
||||
Source(new ServerHistorianOptions { Backend = "QuestDb", Enabled = true }, QuestDbCfg())
|
||||
.SupportedAggregates.ShouldBe(SqlServed, ignoreOrder: true);
|
||||
NullHistorianDataSource.Instance.SupportedAggregates.ShouldBeEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
+201
@@ -0,0 +1,201 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Opc.Ua;
|
||||
using Opc.Ua.Client;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||
using static ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests.ServerEventRig;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 over a real SDK client session: <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) list exactly the aggregates the wired historian
|
||||
/// backend serves — the data sources are built through <see cref="HistorianBackendRegistry"/> exactly as the Host
|
||||
/// builds them — while a live-data <c>AggregateFilter</c> subscription keeps working for every aggregate the SDK
|
||||
/// calculates, advertised or not.
|
||||
/// </summary>
|
||||
public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
{
|
||||
private static readonly string[] GatewayNames = ["Maximum", "Minimum", "TimeAverage", "Total"];
|
||||
|
||||
private static readonly string[] SqlNames =
|
||||
[
|
||||
"Average", "Count", "Delta", "End", "Interpolative", "Maximum", "MaximumActualTime", "Minimum", "MinimumActualTime",
|
||||
"PercentBad", "PercentGood", "Range", "Start", "TimeAverage", "TimeAverage2", "Total", "Total2",
|
||||
];
|
||||
|
||||
private static IHistorianDataSource Backend(string backend)
|
||||
{
|
||||
var config = new ConfigurationBuilder().AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
["ServerHistorian:SqlServer:ConnectionString"] = "Server=localhost;User Id=sa;Password=x;TrustServerCertificate=True",
|
||||
["ServerHistorian:QuestDb:ConnectionString"] = "Host=localhost;Port=18812;Username=admin;Password=quest;Database=qdb",
|
||||
}).Build();
|
||||
var options = new ServerHistorianOptions
|
||||
{
|
||||
Backend = backend, Enabled = true, Endpoint = "http://localhost:5222", ApiKey = "histgw_id_secret", UseTls = false,
|
||||
};
|
||||
// Constructing a data source opens no connection; nothing here reaches a database or a gateway.
|
||||
return HistorianBackendRegistry.Resolve(options, config).CreateDataSource(new ServiceCollection().BuildServiceProvider());
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("Gateway")]
|
||||
[InlineData("SqlServer")]
|
||||
[InlineData("QuestDb")]
|
||||
public async Task Both_folders_advertise_exactly_the_backend_set(string backend)
|
||||
{
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var rig = await StartAsync($"AggAdv{backend}", encrypted: false, ct);
|
||||
try
|
||||
{
|
||||
rig.Nm.HistorianDataSource = Backend(backend);
|
||||
rig.Enforce(false, [Key()]);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
var expected = backend == "Gateway" ? GatewayNames : SqlNames;
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(expected);
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(expected);
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task With_no_historian_wired_both_folders_are_empty_from_boot()
|
||||
{
|
||||
// The Null default is applied by the CreateAggregateManager override, before any source is assigned.
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var rig = await StartAsync("AggAdvNull", encrypted: false, ct);
|
||||
try
|
||||
{
|
||||
rig.Enforce(false, [Key()]);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBeEmpty();
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBeEmpty();
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Rewiring_the_source_resyncs_both_directions_of_the_references()
|
||||
{
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var rig = await StartAsync("AggAdvResync", encrypted: false, ct);
|
||||
try
|
||||
{
|
||||
rig.Enforce(false, [Key()]);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
rig.Nm.HistorianDataSource = Backend("SqlServer");
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(SqlNames);
|
||||
(await ParentsAsync(session, ObjectIds.AggregateFunction_Average, ct)).ShouldBe(
|
||||
[ObjectIds.Server_ServerCapabilities_AggregateFunctions, ObjectIds.HistoryServerCapabilities_AggregateFunctions],
|
||||
ignoreOrder: true);
|
||||
|
||||
// Gateway has no arithmetic Average: dropped from both folders, and its inverse references go with it.
|
||||
rig.Nm.HistorianDataSource = Backend("Gateway");
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(GatewayNames);
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(GatewayNames);
|
||||
(await ParentsAsync(session, ObjectIds.AggregateFunction_Average, ct)).ShouldBeEmpty();
|
||||
|
||||
// And back: removed references are restored.
|
||||
rig.Nm.HistorianDataSource = Backend("QuestDb");
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(SqlNames);
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("StandardDeviationSample")] // served by no historian backend, never advertised
|
||||
[InlineData("Average")] // not served by the Gateway backend wired below
|
||||
public async Task A_live_AggregateFilter_subscription_for_an_unadvertised_aggregate_still_works(string aggregateName)
|
||||
{
|
||||
// The trim touches only the advertised folders, not the AggregateManager's calculator factories, which is what
|
||||
// validates and computes a live-data AggregateFilter. Removing a factory turns this create into
|
||||
// BadAggregateNotSupported.
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var rig = await StartAsync($"AggAdvLive{aggregateName}", encrypted: false, ct);
|
||||
try
|
||||
{
|
||||
rig.Nm.HistorianDataSource = Backend("Gateway");
|
||||
rig.Enforce(false, [Key()]);
|
||||
var id = rig.EnsureRawVariable("agg", "Double", writable: false);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
var aggregateId = (NodeId)typeof(ObjectIds).GetField("AggregateFunction_" + aggregateName)!.GetValue(null)!;
|
||||
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct))
|
||||
.ShouldNotContain(aggregateName);
|
||||
|
||||
var subscription = new Subscription(session.DefaultSubscription) { PublishingInterval = 100 };
|
||||
session.AddSubscription(subscription);
|
||||
await subscription.CreateAsync(ct);
|
||||
var item = new MonitoredItem(subscription.DefaultItem)
|
||||
{
|
||||
StartNodeId = ExpandedNodeId.ToNodeId(ExpandedNodeId.Parse(RawNsu(id)), session.NamespaceUris),
|
||||
AttributeId = Attributes.Value,
|
||||
SamplingInterval = 100,
|
||||
QueueSize = 10,
|
||||
Filter = new AggregateFilter
|
||||
{
|
||||
AggregateType = aggregateId,
|
||||
ProcessingInterval = 1000,
|
||||
StartTime = DateTime.UtcNow,
|
||||
},
|
||||
};
|
||||
var notifications = 0;
|
||||
item.Notification += (mi, _) => Interlocked.Add(ref notifications, mi.DequeueValues().Count);
|
||||
subscription.AddItem(item);
|
||||
await subscription.ApplyChangesAsync(ct);
|
||||
|
||||
item.Status.Error.ShouldBeNull($"AggregateFilter({aggregateName}) must be accepted on live data");
|
||||
// And it is computed: the SDK calculator emits one value per processing interval.
|
||||
var deadline = DateTime.UtcNow.AddSeconds(15);
|
||||
while (Volatile.Read(ref notifications) == 0 && DateTime.UtcNow < deadline) await Task.Delay(100, ct);
|
||||
Volatile.Read(ref notifications).ShouldBeGreaterThan(0, "the live aggregate must produce interval values");
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>Forward Organizes children of an AggregateFunctions folder, by BrowseName, sorted.</summary>
|
||||
private static async Task<List<string>> AdvertisedAsync(ISession session, NodeId folder, CancellationToken ct)
|
||||
{
|
||||
var refs = await BrowseAsync(session, folder, BrowseDirection.Forward, ct);
|
||||
return refs.Select(r => r.BrowseName.Name).OrderBy(n => n, StringComparer.Ordinal).ToList();
|
||||
}
|
||||
|
||||
/// <summary>The folders an aggregate-function node is organized under (inverse Organizes).</summary>
|
||||
private static async Task<List<NodeId>> ParentsAsync(ISession session, NodeId aggregate, CancellationToken ct)
|
||||
{
|
||||
var refs = await BrowseAsync(session, aggregate, BrowseDirection.Inverse, ct);
|
||||
return refs.Select(r => ExpandedNodeId.ToNodeId(r.NodeId, session.NamespaceUris)).ToList();
|
||||
}
|
||||
|
||||
private static async Task<ReferenceDescriptionCollection> BrowseAsync(
|
||||
ISession session, NodeId node, BrowseDirection direction, CancellationToken ct)
|
||||
{
|
||||
var response = await session.BrowseAsync(
|
||||
requestHeader: null,
|
||||
view: null,
|
||||
requestedMaxReferencesPerNode: 0,
|
||||
nodesToBrowse:
|
||||
[
|
||||
new BrowseDescription
|
||||
{
|
||||
NodeId = node,
|
||||
BrowseDirection = direction,
|
||||
ReferenceTypeId = ReferenceTypeIds.Organizes,
|
||||
IncludeSubtypes = false,
|
||||
NodeClassMask = 0,
|
||||
ResultMask = (uint)BrowseResultMask.All,
|
||||
},
|
||||
],
|
||||
ct: ct);
|
||||
StatusCode.IsGood(response.Results[0].StatusCode).ShouldBeTrue($"browse {node}: {response.Results[0].StatusCode}");
|
||||
return response.Results[0].References;
|
||||
}
|
||||
}
|
||||
@@ -278,6 +278,54 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #682: an aggregate the server maps but the wired backend does not declare in
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/> is refused <c>BadAggregateNotSupported</c> UP FRONT — the
|
||||
/// backend is never called (the same set is what the AggregateFunctions folders advertise). The Null default
|
||||
/// declares none, so a processed read with no historian is refused too.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Processed_aggregate_outside_the_backend_set_is_refused_without_a_backend_call()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
var nm = server.NodeManager!;
|
||||
var fake = new RecordingHistorianDataSource
|
||||
{
|
||||
ProcessedResult = Empty(),
|
||||
Supported = new HashSet<HistoryAggregateType> { HistoryAggregateType.TimeAverage },
|
||||
};
|
||||
nm.HistorianDataSource = fake;
|
||||
|
||||
nm.EnsureVariable("eq-1/gw", parentFolderNodeId: null, displayName: "Gw", dataType: "Float",
|
||||
writable: false, historianTagname: "WW.Gw", realm: AddressSpaceRealm.Uns);
|
||||
var nodeId = nm.TryGetVariable("eq-1/gw")!.NodeId;
|
||||
|
||||
ReadProcessedDetails Details(NodeId aggregate) => new()
|
||||
{
|
||||
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||
EndTime = DateTime.UtcNow,
|
||||
ProcessingInterval = 10_000.0,
|
||||
AggregateType = new NodeIdCollection { aggregate },
|
||||
};
|
||||
|
||||
// Average is mapped by the server but not in this backend's set ⇒ refused, no call.
|
||||
InvokeHistoryRead(server, nm, Details(ObjectIds.AggregateFunction_Average), nodeId).Item2[0].StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadAggregateNotSupported);
|
||||
fake.LastCall.ShouldBeNull();
|
||||
|
||||
// Positive control: a member of the set reaches the backend.
|
||||
InvokeHistoryRead(server, nm, Details(ObjectIds.AggregateFunction_TimeAverage), nodeId);
|
||||
fake.LastCall.ShouldBe("Processed");
|
||||
fake.LastAggregate.ShouldBe(HistoryAggregateType.TimeAverage);
|
||||
|
||||
// No historian: the Null default serves no aggregate.
|
||||
nm.HistorianDataSource = NullHistorianDataSource.Instance;
|
||||
InvokeHistoryRead(server, nm, Details(ObjectIds.AggregateFunction_TimeAverage), nodeId).Item2[0].StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadAggregateNotSupported);
|
||||
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #677: a client <c>AggregateConfiguration</c> that differs from the server defaults is refused per node with
|
||||
/// <c>BadAggregateConfigurationRejected</c> (it used to be silently ignored); the source is never invoked.
|
||||
@@ -626,6 +674,8 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
public bool ThrowOnRead { get; init; }
|
||||
public bool AggregateNotSupported { get; init; }
|
||||
public bool LimitExceeded { get; init; }
|
||||
public IReadOnlySet<HistoryAggregateType> Supported { get; init; } = HistoryAggregateTypes.All;
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => Supported;
|
||||
public HistorianRead RawResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
public HistorianRead ProcessedResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
public HistorianRead AtTimeResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
|
||||
Reference in New Issue
Block a user