fix(historian): trim only HistoryServerCapabilities/AggregateFunctions; null set = all (#682 review)
ServerCapabilities/AggregateFunctions (i=2997) is Part 5's list of every aggregate the server supports, and live AggregateFilter subscriptions still compute all 37 SDK aggregates, so it keeps the SDK's full set; only i=11201 follows the backend. A null SupportedAggregates is treated as the interface default (all) by both the advertisement and HistoryRead-Processed (it used to NRE inside the batch). Docs: i=2997/i=11201 split and the boot window. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
@@ -1353,7 +1353,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (10, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted seven (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM` — plus the **duration-based** PercentGood/PercentBad, #680) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`, percentages included). ⚠️ **#680 status rules (client-visible):** TimeAverage/Total are Uncertain if any part of the interval uses an Uncertain value, is bridged across a Bad sample, is extrapolated past the last sample (⇒ a live tag's latest interval is Uncertain) or is not covered; TimeAverage2/Total2 keep the SDK time-based rule (Uncertain time counts as Good) but never extrapolate past the last sample — that interval is Partial, later ones `BadNoData` (a report-by-exception tag with nothing after its last sample reads `BadNoData` until its next sample arrives). `docs/Historian.md` §"Time-weighted aggregates". ⚠️ **#680 per-sample (client-visible):** `Count` counts **Good** samples only (value-based status; an all-Bad interval is generic Bad, value null) and an empty interval **inside the tag's data range** is `0`/Good (outside it still `BadNoData`); `Start`/`End` return the **raw** first/last sample of any quality — its own timestamp and status, value null when Bad; `Delta` is Uncertain only when the interval's first or last sample is unusable; and every per-sample aggregate but `Average` sets **end-of-data Partial** on the interval holding the last sample (forward incl. its earlier edge, reverse strictly inside), never on a Bad result. The data-range lookup is a seek on SQL Server but a partition scan on QuestDB (unindexed `tag` SYMBOL). `docs/Historian.md` §"Aggregates". Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention.
|
||||
- **QuestDB** (`Driver.Historian.QuestDb`, `ServerHistorian:QuestDb:*`, Npgsql over PG wire; docker-dev overlay `docker-compose.questdb.yml`): same kinds/status/aggregates/HistoryUpdate as SqlServer, soft delete by tombstone (INSERT only — never `UPDATE`/`DELETE`; every read filters `NOT deleted`). ⚠️ **Timestamps are µs** (`QuestDbConnection.Round`/`BindTimestamp`; `IHistorianDataSource.TimestampResolutionTicks` = 10 steps HistoryRead paging), **pre-1970 cannot be stored** (writer drops, HistoryUpdate `BadOutOfRange`), every write waits for WAL visibility, TTL is enforced on write commits only, set Npgsql `Maximum Pool Size` < QuestDB's 64. `docs/Historian.md` §"QuestDB backend".
|
||||
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
|
||||
- **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**) is the one authority for both `AggregateFunctions` folders (i=2997, i=11201 — only the folder references are trimmed; the SDK's calculator factories stay for live `AggregateFilter` subscriptions) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates.
|
||||
- **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**; a null set ⇒ all) is the one authority for `HistoryServerCapabilities/AggregateFunctions` (i=11201 — only its folder references are trimmed) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ `ServerCapabilities/AggregateFunctions` (i=2997) is deliberately NOT trimmed: it keeps the SDK's 37 because live `AggregateFilter` subscriptions still compute them all. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates.
|
||||
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
||||
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
|
||||
- **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization AND after planning — an unservable request (Remove / `IsDeleteModified` / not historized) stays `BadHistoryOperationUnsupported` off-Primary; the verdict is `IHistoryUpdateRoleView` = the data-plane verdict but **closed until this node's first redundancy snapshot**, so a pair cannot both accept in the boot window; admin-only nodes register no view ⇒ no gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ node-level `BadHistoryOperationUnsupported` on every arm (the updater throws `NotSupportedException`, nothing sent; the `HistoryWrite` bit is still advertised — not knowable at materialise time), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key).
|
||||
|
||||
+30
-19
@@ -96,7 +96,7 @@ HistoryRead is served from any node (the database is shared). HistoryUpdate is *
|
||||
The `ServerHistorian` section in `appsettings.json` controls the historian read path.
|
||||
`Enabled` defaults to `false`; when disabled, the server registers `NullHistorianDataSource`
|
||||
and Raw / AtTime / Events HistoryRead calls on historized nodes return `GoodNoData` (empty, not an error); a
|
||||
Processed read is `BadAggregateNotSupported` and no aggregate functions are advertised
|
||||
Processed read is `BadAggregateNotSupported` and no history aggregate functions are advertised
|
||||
([#682](#advertised-aggregates-aggregatefunctions-682)).
|
||||
|
||||
```json
|
||||
@@ -778,34 +778,45 @@ NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopul
|
||||
Each backend declares the aggregates it serves as `IHistorianDataSource.SupportedAggregates` — **the single authority** for
|
||||
two things:
|
||||
|
||||
- **What the server advertises.** `Server/ServerCapabilities/AggregateFunctions` (`i=2997`) and
|
||||
`HistoryServerCapabilities/AggregateFunctions` (`i=11201`) — two separate folder nodes that organize the same
|
||||
aggregate-function objects — list exactly that set. Before #682 both listed the SDK's full 37 regardless of backend.
|
||||
- **What the server advertises as HISTORY aggregates.** `HistoryServerCapabilities/AggregateFunctions` (`i=11201`) lists
|
||||
exactly that set. Before #682 it listed the SDK's full 37 regardless of backend.
|
||||
- **What HistoryRead-Processed refuses up front.** An aggregate outside the set is `BadAggregateNotSupported` for that node
|
||||
without a backend call (the backend still throws `HistoryAggregateNotSupportedException` for one as defence in depth —
|
||||
derived from the same table, never a second list).
|
||||
derived from the same table, never a second list). A source whose `SupportedAggregates` is `null` (out-of-tree, against
|
||||
the contract) is treated as the interface default — **all** — by both consumers.
|
||||
|
||||
| `ServerHistorian:Backend` | Advertised | Declared by |
|
||||
|---|---|---|
|
||||
| `Gateway` | 4 — TimeAverage, Minimum, Maximum, Total | `GatewayHistorianDataSource.ServedAggregates` (today `AggregateModeMapper.Served`, derived from its mapping switch) |
|
||||
| `SqlServer`, `QuestDb` | 17 — the [aggregate table](#aggregates-processed-reads) | `AggregateFamilies.Served` (Sql.Common; every member with a family) |
|
||||
| historian disabled (Null source) | **none** | `NullHistorianDataSource` |
|
||||
⚠️ **`Server/ServerCapabilities/AggregateFunctions` (`i=2997`) is deliberately NOT trimmed** — it keeps the SDK's full 37 in
|
||||
every configuration, including with no historian. Part 5 defines it as every aggregate the *server* supports, and the server
|
||||
still computes all of them for live data: a MonitoredItem `AggregateFilter` is validated and computed by the SDK's
|
||||
`AggregateManager`, not the historian. The SDK models the same split (`DiagnosticsNodeManager.AddAggregateFunction` always adds
|
||||
to i=2997 and to i=11201 only for historical aggregates). The two are separate folder nodes organizing the same
|
||||
aggregate-function objects. A client choosing a history aggregate should read **i=11201**.
|
||||
|
||||
| `ServerHistorian:Backend` | i=11201 (history) | i=2997 (server) | Declared by |
|
||||
|---|---|---|---|
|
||||
| `Gateway` | 4 — TimeAverage, Minimum, Maximum, Total | all 37 | `GatewayHistorianDataSource.ServedAggregates` (today `AggregateModeMapper.Served`, derived from its mapping switch) |
|
||||
| `SqlServer`, `QuestDb` | 17 — the [aggregate table](#aggregates-processed-reads) | all 37 | `AggregateFamilies.Served` (Sql.Common; every member with a family) |
|
||||
| historian disabled (Null source) | **none** | all 37 | `NullHistorianDataSource` |
|
||||
|
||||
⚠️ **Client-visible: with no historian configured a processed HistoryRead is now `BadAggregateNotSupported`, not
|
||||
`GoodNoData`.** An aggregate is a statistic the server cannot compute without a history, and the empty folders now say so;
|
||||
`GoodNoData`.** An aggregate is a statistic the server cannot compute without a history, and the empty i=11201 now says so;
|
||||
Raw / AtTime / Events reads on the Null source still return the empty `GoodNoData` result.
|
||||
|
||||
How it is wired: `OtOpcUaSdkServer` keeps the SDK's `CreateAggregateManager` (all 37 calculator factories) and then trims
|
||||
only the folder ⇄ aggregate `Organizes` references to the node manager's current source — the Null default at boot; the
|
||||
Host's `SetHistorianDataSource` (the backend resolved once from `ServerHistorian:Backend`) re-syncs it right after start,
|
||||
and every later assignment re-syncs both directions. ⚠️ **The calculator factories are deliberately NOT removed**: a
|
||||
live-data MonitoredItem with an `AggregateFilter` is validated and computed by the SDK's `AggregateManager`, not the
|
||||
historian, so live aggregate subscriptions keep working for every SDK aggregate, advertised or not (pinned over the wire
|
||||
by `AdvertisedAggregateFunctionsWireTests`). The aggregate-function object nodes themselves stay readable. A boot log line
|
||||
names the advertised set.
|
||||
only the i=11201 ⇄ aggregate `Organizes` references (both directions) to the node manager's current source — the Null
|
||||
default at boot; the Host's `SetHistorianDataSource` (the backend resolved once from `ServerHistorian:Backend`) re-syncs it
|
||||
right after start, and every later assignment re-syncs both ways. ⚠️ **The calculator factories are deliberately NOT
|
||||
removed**: live aggregate subscriptions keep working for every SDK aggregate (pinned over the wire by
|
||||
`AdvertisedAggregateFunctionsWireTests`). The aggregate-function object nodes themselves stay readable. A boot log line
|
||||
names the advertised history set.
|
||||
|
||||
⚠️ **Boot window.** `SetHistorianDataSource` runs right after the SDK server has started (`OtOpcUaServerHostedService.StartAsync`),
|
||||
so for the few milliseconds between the endpoint opening and that call i=11201 is empty and a processed HistoryRead is
|
||||
refused `BadAggregateNotSupported` — the Null default's answer. A client that browses i=11201 in that window sees an empty
|
||||
list; re-browse after connect if it matters.
|
||||
|
||||
Not changed: the `HistoryServerCapabilities` booleans (`AccessHistoryDataCapability`, `InsertDataCapability`, …) are still
|
||||
whatever the SDK sets — #682 touches only the two folders (follow-up candidate).
|
||||
whatever the SDK sets — #682 touches only the i=11201 folder (follow-up candidate).
|
||||
|
||||
### Known limitations
|
||||
|
||||
|
||||
@@ -176,9 +176,10 @@ public interface IHistorianDataSource : IDisposable
|
||||
|
||||
/// <summary>
|
||||
/// The Part 13 aggregates this backend serves for a processed HistoryRead (Gitea #682) — the ONE authority for
|
||||
/// both what the server advertises under <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201), and what its HistoryRead-Processed answers
|
||||
/// <c>BadAggregateNotSupported</c> up front without calling the backend.
|
||||
/// both what the server advertises under <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) and what
|
||||
/// its HistoryRead-Processed answers <c>BadAggregateNotSupported</c> up front without calling the backend.
|
||||
/// (<c>ServerCapabilities/AggregateFunctions</c>, i=2997, is not affected: it lists every aggregate the server
|
||||
/// computes, including for live <c>AggregateFilter</c> subscriptions.)
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The default is <see cref="HistoryAggregateTypes.All"/> — "every aggregate the server can name; a member I cannot
|
||||
@@ -187,7 +188,8 @@ public interface IHistorianDataSource : IDisposable
|
||||
/// <c>null</c> "all" sentinel so every consumer handles one shape. Every PRODUCTION backend declares its set itself
|
||||
/// (a reflection guard pins that): a wrapper that forgot to forward this would otherwise advertise every aggregate.
|
||||
/// The set is read once when the source is wired (the backend is fixed at boot by <c>ServerHistorian:Backend</c>)
|
||||
/// and again on every processed read, so it must be cheap and stable — return a static frozen set.
|
||||
/// and again on every processed read, so it must be cheap and stable — return a static frozen set. A source that
|
||||
/// returns <c>null</c> anyway is treated as the default (all) by both consumers.
|
||||
/// </remarks>
|
||||
IReadOnlySet<HistoryAggregateType> SupportedAggregates => HistoryAggregateTypes.All;
|
||||
|
||||
|
||||
@@ -6,24 +6,23 @@ using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — make the two advertised aggregate-function folders, <c>ServerCapabilities/AggregateFunctions</c>
|
||||
/// (i=2997) and <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201), list exactly the aggregates the wired
|
||||
/// historian backend serves (<see cref="IHistorianDataSource.SupportedAggregates"/>), instead of the SDK's full
|
||||
/// registered set.
|
||||
/// Gitea #682 — make <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) list exactly the aggregates the
|
||||
/// wired historian backend serves for HistoryRead (<see cref="IHistorianDataSource.SupportedAggregates"/>), instead of
|
||||
/// the SDK's full registered set.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// The SDK's <c>AggregateManager.RegisterFactory</c> both registers a calculator factory AND adds an
|
||||
/// <c>Organizes</c> reference from each folder to the aggregate's object node. Only the REFERENCES are trimmed
|
||||
/// here: the factories stay, because a live-data MonitoredItem with an <c>AggregateFilter</c> is validated
|
||||
/// against (and computed by) the AggregateManager, not the historian — removing a factory would break live
|
||||
/// aggregate subscriptions for every aggregate the historian happens not to serve.
|
||||
/// ⚠️ <c>ServerCapabilities/AggregateFunctions</c> (i=2997) is deliberately NOT trimmed. Part 5 defines it as every
|
||||
/// aggregate the SERVER supports, and the server still supports all the SDK's registered aggregates for live
|
||||
/// data: a MonitoredItem <c>AggregateFilter</c> is validated and computed by the SDK's <c>AggregateManager</c>,
|
||||
/// not the historian. The SDK itself models the split — <c>DiagnosticsNodeManager.AddAggregateFunction</c> always
|
||||
/// adds to i=2997 and adds to i=11201 only when the aggregate is historical. i=11201 is the history surface, so it
|
||||
/// is the one that follows the backend.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The two folders are separate nodes that reference the same aggregate object nodes (both directions). The
|
||||
/// aggregate nodes themselves are kept (they are standard type-system nodes a client may still Read); only the
|
||||
/// folder ⇄ aggregate <c>Organizes</c> pair is removed or restored. Idempotent and reversible, so re-wiring a
|
||||
/// different source re-syncs both folders.
|
||||
/// Only the folder ⇄ aggregate <c>Organizes</c> reference pair is removed or restored (both directions); the
|
||||
/// calculator factories and the aggregate-function object nodes stay. Idempotent and reversible, so re-wiring a
|
||||
/// different source re-syncs the folder.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class AdvertisedAggregateFunctions
|
||||
@@ -35,20 +34,26 @@ internal static class AdvertisedAggregateFunctions
|
||||
.Select(f => (NodeId)f.GetValue(null)!)
|
||||
.ToArray();
|
||||
|
||||
/// <summary>The two advertised folders (both are trimmed — they are separate nodes).</summary>
|
||||
internal static readonly NodeId[] Folders =
|
||||
[
|
||||
ObjectIds.Server_ServerCapabilities_AggregateFunctions,
|
||||
ObjectIds.HistoryServerCapabilities_AggregateFunctions,
|
||||
];
|
||||
/// <summary>The trimmed folder — the HISTORY aggregate list only (see remarks for why not i=2997).</summary>
|
||||
internal static readonly NodeId Folder = ObjectIds.HistoryServerCapabilities_AggregateFunctions;
|
||||
|
||||
/// <summary>
|
||||
/// Sync both folders to <paramref name="served"/>: an aggregate-function node is organized under a folder iff the
|
||||
/// server maps it (<see cref="AggregateFunctionMap"/>) AND <paramref name="served"/> contains it.
|
||||
/// The set a source serves, treating a <c>null</c> <see cref="IHistorianDataSource.SupportedAggregates"/> (an
|
||||
/// out-of-tree source that returns null despite the non-nullable contract) as <see cref="HistoryAggregateTypes.All"/>,
|
||||
/// the interface default — the ONE null rule both the advertisement and HistoryRead-Processed use.
|
||||
/// </summary>
|
||||
/// <param name="source">The historian source.</param>
|
||||
/// <returns>Its served set, never null.</returns>
|
||||
public static IReadOnlySet<HistoryAggregateType> ServedBy(IHistorianDataSource source) =>
|
||||
source.SupportedAggregates ?? HistoryAggregateTypes.All;
|
||||
|
||||
/// <summary>
|
||||
/// Sync i=11201 to <paramref name="served"/>: an aggregate-function node is organized under it iff the server maps
|
||||
/// it (<see cref="AggregateFunctionMap"/>) AND <paramref name="served"/> contains it.
|
||||
/// </summary>
|
||||
/// <param name="server">The running server.</param>
|
||||
/// <param name="served">The backend's served aggregates.</param>
|
||||
/// <returns>The advertised NodeIds, or <c>null</c> when the diagnostics node manager is not available yet.</returns>
|
||||
/// <returns>The advertised NodeIds, or <c>null</c> when the diagnostics node manager or folder is not available.</returns>
|
||||
public static IReadOnlyList<NodeId>? Apply(IServerInternal server, IReadOnlySet<HistoryAggregateType> served)
|
||||
{
|
||||
if (server?.DiagnosticsNodeManager is not CustomNodeManager2 diagnostics) return null;
|
||||
@@ -56,10 +61,8 @@ internal static class AdvertisedAggregateFunctions
|
||||
var advertised = new List<NodeId>();
|
||||
lock (diagnostics.Lock)
|
||||
{
|
||||
var folders = Folders
|
||||
.Select(id => diagnostics.FindPredefinedNode(id, typeof(BaseObjectState)))
|
||||
.Where(f => f is not null)
|
||||
.ToList();
|
||||
var folder = diagnostics.FindPredefinedNode(Folder, typeof(BaseObjectState));
|
||||
if (folder is null) return null;
|
||||
|
||||
foreach (var aggregateId in StandardAggregateIds)
|
||||
{
|
||||
@@ -69,19 +72,16 @@ internal static class AdvertisedAggregateFunctions
|
||||
var advertise = AggregateFunctionMap.Map(aggregateId) is { } aggregate && served.Contains(aggregate);
|
||||
if (advertise) advertised.Add(aggregateId);
|
||||
|
||||
foreach (var folder in folders)
|
||||
var linked = folder.ReferenceExists(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
if (advertise && !linked)
|
||||
{
|
||||
var linked = folder!.ReferenceExists(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
if (advertise && !linked)
|
||||
{
|
||||
folder.AddReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.AddReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
else if (!advertise && linked)
|
||||
{
|
||||
folder.RemoveReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.RemoveReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
folder.AddReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.AddReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
else if (!advertise && linked)
|
||||
{
|
||||
folder.RemoveReference(ReferenceTypeIds.Organizes, false, aggregateId);
|
||||
aggregateNode.RemoveReference(ReferenceTypeIds.Organizes, true, folder.NodeId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -345,7 +345,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
/// node-manager <c>Lock</c>, so the override may block-bridge to this (async) source.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Gitea #682: assigning a source also re-syncs the advertised <c>AggregateFunctions</c> folders to its
|
||||
/// Gitea #682: assigning a source also re-syncs <c>HistoryServerCapabilities/AggregateFunctions</c> to its
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/> (see <see cref="SyncAdvertisedAggregates"/>).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
@@ -360,9 +360,10 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — trim <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) to the current source's
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/>. Called on every <see cref="HistorianDataSource"/>
|
||||
/// Gitea #682 — trim <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) to the current source's
|
||||
/// <see cref="IHistorianDataSource.SupportedAggregates"/> (null ⇒ all, <see cref="AdvertisedAggregateFunctions.ServedBy"/>).
|
||||
/// <c>ServerCapabilities/AggregateFunctions</c> (i=2997) keeps the SDK's full set — the server still computes every
|
||||
/// one of them for live <c>AggregateFilter</c> subscriptions. Called on every <see cref="HistorianDataSource"/>
|
||||
/// assignment and by <see cref="OtOpcUaSdkServer"/> right after the SDK registers its aggregate factories, so the
|
||||
/// Null default is applied from boot. Only the folder references change — the AggregateManager's calculator
|
||||
/// factories (live <c>AggregateFilter</c> subscriptions) are untouched. Never throws: a failure leaves the previous
|
||||
@@ -373,11 +374,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
try
|
||||
{
|
||||
var source = _historianDataSource;
|
||||
var advertised = AdvertisedAggregateFunctions.Apply(Server, source.SupportedAggregates);
|
||||
var advertised = AdvertisedAggregateFunctions.Apply(Server, AdvertisedAggregateFunctions.ServedBy(source));
|
||||
if (advertised is not null)
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"OtOpcUaNodeManager: advertising {Count} aggregate function(s) for historian {Source}: {Aggregates}",
|
||||
"OtOpcUaNodeManager: advertising {Count} history aggregate function(s) for historian {Source}: {Aggregates}",
|
||||
advertised.Count, source.GetType().Name,
|
||||
string.Join(", ", advertised.Select(id => AggregateFunctionMap.Map(id)?.ToString() ?? id.ToString())));
|
||||
}
|
||||
@@ -5091,7 +5092,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// #677: a client AggregateConfiguration other than the server defaults is refused per node, never silently ignored.
|
||||
var configurationServable = ServerAggregateConfiguration.IsServable(details.AggregateConfiguration);
|
||||
nodesToProcess = AuthorizeHistoryRead(context, nodesToProcess, results, errors);
|
||||
var source = HistorianDataSource;
|
||||
var served = AdvertisedAggregateFunctions.ServedBy(HistorianDataSource);
|
||||
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||
{
|
||||
using var deadline = CreateDeadline();
|
||||
@@ -5102,11 +5103,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// AggregateType is a per-node parallel collection (same length as nodesToRead, enforced by
|
||||
// the base dispatcher). handle.Index is the node's position in that collection.
|
||||
var aggregateNodeId = details.AggregateType[handle.Index];
|
||||
// #682: one authority — the source's SupportedAggregates (also what the AggregateFunctions folders
|
||||
// advertise). An aggregate the server cannot name, or the backend does not serve, is refused up front
|
||||
// without a backend call.
|
||||
// #682: one authority — the source's SupportedAggregates (also what HistoryServerCapabilities/
|
||||
// AggregateFunctions advertises; null ⇒ all, the same rule). An aggregate the server cannot name, or the
|
||||
// backend does not serve, is refused up front without a backend call.
|
||||
var aggregate = AggregateFunctionMap.Map(aggregateNodeId);
|
||||
if (aggregate is null || !source.SupportedAggregates.Contains(aggregate.Value))
|
||||
if (aggregate is null || !served.Contains(aggregate.Value))
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadAggregateNotSupported;
|
||||
continue;
|
||||
|
||||
@@ -202,9 +202,10 @@ public sealed class OtOpcUaSdkServer : StandardServer
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 — keep the SDK's full set of calculator factories (live-data <c>AggregateFilter</c> subscriptions
|
||||
/// need them), then trim the advertised <c>AggregateFunctions</c> folders to the node manager's current historian
|
||||
/// source. At this point that is the Null default (no aggregates); the Host's
|
||||
/// <see cref="SetHistorianDataSource"/> re-syncs to the configured backend's set right after start.
|
||||
/// need them, and <c>ServerCapabilities/AggregateFunctions</c> keeps listing all of them), then trim
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> to the node manager's current historian source. At this
|
||||
/// point that is the Null default (no history aggregates); the Host's <see cref="SetHistorianDataSource"/>
|
||||
/// re-syncs to the configured backend's set right after start.
|
||||
/// </summary>
|
||||
/// <inheritdoc />
|
||||
protected override AggregateManager CreateAggregateManager(
|
||||
|
||||
+30
-15
@@ -12,11 +12,11 @@ using static ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests.ServerEventRig;
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #682 over a real SDK client session: <c>ServerCapabilities/AggregateFunctions</c> (i=2997) and
|
||||
/// <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) list exactly the aggregates the wired historian
|
||||
/// backend serves — the data sources are built through <see cref="HistorianBackendRegistry"/> exactly as the Host
|
||||
/// builds them — while a live-data <c>AggregateFilter</c> subscription keeps working for every aggregate the SDK
|
||||
/// calculates, advertised or not.
|
||||
/// Gitea #682 over a real SDK client session: <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) lists
|
||||
/// exactly the aggregates the wired historian backend serves — the data sources are built through
|
||||
/// <see cref="HistorianBackendRegistry"/> exactly as the Host builds them — while
|
||||
/// <c>ServerCapabilities/AggregateFunctions</c> (i=2997) keeps the SDK's full registered set in every configuration,
|
||||
/// because a live-data <c>AggregateFilter</c> subscription keeps working for every one of them.
|
||||
/// </summary>
|
||||
public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
{
|
||||
@@ -28,6 +28,17 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
"PercentBad", "PercentGood", "Range", "Start", "TimeAverage", "TimeAverage2", "Total", "Total2",
|
||||
];
|
||||
|
||||
/// <summary>The SDK's 37 registered aggregates (<c>StandardServer.CreateAggregateManager</c>) — i=2997, always.</summary>
|
||||
private static readonly string[] AllSdkNames =
|
||||
[
|
||||
"AnnotationCount", "Average", "Count", "Delta", "DeltaBounds", "DurationBad", "DurationGood", "DurationInStateNonZero",
|
||||
"DurationInStateZero", "End", "EndBound", "Interpolative", "Maximum", "Maximum2", "MaximumActualTime",
|
||||
"MaximumActualTime2", "Minimum", "Minimum2", "MinimumActualTime", "MinimumActualTime2", "NumberOfTransitions",
|
||||
"PercentBad", "PercentGood", "Range", "Range2", "StandardDeviationPopulation", "StandardDeviationSample", "Start",
|
||||
"StartBound", "TimeAverage", "TimeAverage2", "Total", "Total2", "VariancePopulation", "VarianceSample",
|
||||
"WorstQuality", "WorstQuality2",
|
||||
];
|
||||
|
||||
private static IHistorianDataSource Backend(string backend)
|
||||
{
|
||||
var config = new ConfigurationBuilder().AddInMemoryCollection(new Dictionary<string, string?>
|
||||
@@ -47,7 +58,7 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
[InlineData("Gateway")]
|
||||
[InlineData("SqlServer")]
|
||||
[InlineData("QuestDb")]
|
||||
public async Task Both_folders_advertise_exactly_the_backend_set(string backend)
|
||||
public async Task History_folder_advertises_the_backend_set_and_the_server_folder_keeps_every_aggregate(string backend)
|
||||
{
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var rig = await StartAsync($"AggAdv{backend}", encrypted: false, ct);
|
||||
@@ -58,14 +69,14 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
var expected = backend == "Gateway" ? GatewayNames : SqlNames;
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(expected);
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(expected);
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(AllSdkNames);
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task With_no_historian_wired_both_folders_are_empty_from_boot()
|
||||
public async Task With_no_historian_wired_the_history_folder_is_empty_from_boot()
|
||||
{
|
||||
// The Null default is applied by the CreateAggregateManager override, before any source is assigned.
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
@@ -75,8 +86,8 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
rig.Enforce(false, [Key()]);
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBeEmpty();
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBeEmpty();
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(AllSdkNames);
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
@@ -92,16 +103,18 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
|
||||
rig.Nm.HistorianDataSource = Backend("SqlServer");
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(SqlNames);
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(SqlNames);
|
||||
(await ParentsAsync(session, ObjectIds.AggregateFunction_Average, ct)).ShouldBe(
|
||||
[ObjectIds.Server_ServerCapabilities_AggregateFunctions, ObjectIds.HistoryServerCapabilities_AggregateFunctions],
|
||||
ignoreOrder: true);
|
||||
|
||||
// Gateway has no arithmetic Average: dropped from both folders, and its inverse references go with it.
|
||||
// Gateway has no arithmetic Average: dropped from the history folder (inverse reference too), kept in the
|
||||
// server folder.
|
||||
rig.Nm.HistorianDataSource = Backend("Gateway");
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(GatewayNames);
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct)).ShouldBe(GatewayNames);
|
||||
(await ParentsAsync(session, ObjectIds.AggregateFunction_Average, ct)).ShouldBeEmpty();
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct)).ShouldBe(AllSdkNames);
|
||||
(await ParentsAsync(session, ObjectIds.AggregateFunction_Average, ct)).ShouldBe(
|
||||
[ObjectIds.Server_ServerCapabilities_AggregateFunctions]);
|
||||
|
||||
// And back: removed references are restored.
|
||||
rig.Nm.HistorianDataSource = Backend("QuestDb");
|
||||
@@ -111,7 +124,7 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("StandardDeviationSample")] // served by no historian backend, never advertised
|
||||
[InlineData("StandardDeviationSample")] // served by no historian backend, never in the history folder
|
||||
[InlineData("Average")] // not served by the Gateway backend wired below
|
||||
public async Task A_live_AggregateFilter_subscription_for_an_unadvertised_aggregate_still_works(string aggregateName)
|
||||
{
|
||||
@@ -128,8 +141,10 @@ public sealed class AdvertisedAggregateFunctionsWireTests
|
||||
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||
var aggregateId = (NodeId)typeof(ObjectIds).GetField("AggregateFunction_" + aggregateName)!.GetValue(null)!;
|
||||
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct))
|
||||
(await AdvertisedAsync(session, ObjectIds.HistoryServerCapabilities_AggregateFunctions, ct))
|
||||
.ShouldNotContain(aggregateName);
|
||||
(await AdvertisedAsync(session, ObjectIds.Server_ServerCapabilities_AggregateFunctions, ct))
|
||||
.ShouldContain(aggregateName); // i=2997 tells the truth: the live aggregate IS supported
|
||||
|
||||
var subscription = new Subscription(session.DefaultSubscription) { PublishingInterval = 100 };
|
||||
session.AddSubscription(subscription);
|
||||
|
||||
@@ -326,6 +326,48 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #682 review: an out-of-tree source returning a NULL <see cref="IHistorianDataSource.SupportedAggregates"/> (despite
|
||||
/// the non-nullable contract) is treated as the interface default — ALL — by BOTH consumers: the processed read
|
||||
/// dispatches (no NullReferenceException failing the whole request) and HistoryServerCapabilities/AggregateFunctions
|
||||
/// advertises every mapped aggregate.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task A_null_supported_set_means_all_for_both_the_read_and_the_advertisement()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
var nm = server.NodeManager!;
|
||||
var fake = new RecordingHistorianDataSource { ProcessedResult = Empty(), Supported = null };
|
||||
nm.HistorianDataSource = fake;
|
||||
|
||||
nm.EnsureVariable("eq-1/nul", parentFolderNodeId: null, displayName: "Nul", dataType: "Float",
|
||||
writable: false, historianTagname: "WW.Nul", realm: AddressSpaceRealm.Uns);
|
||||
var nodeId = nm.TryGetVariable("eq-1/nul")!.NodeId;
|
||||
|
||||
var (_, errors) = InvokeHistoryRead(server, nm, new ReadProcessedDetails
|
||||
{
|
||||
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||
EndTime = DateTime.UtcNow,
|
||||
ProcessingInterval = 10_000.0,
|
||||
AggregateType = new NodeIdCollection { ObjectIds.AggregateFunction_StandardDeviationPopulation },
|
||||
}, nodeId);
|
||||
StatusCode.IsBad(errors[0].StatusCode).ShouldBeFalse($"got {errors[0].StatusCode}");
|
||||
fake.LastCall.ShouldBe("Processed");
|
||||
fake.LastAggregate.ShouldBe(HistoryAggregateType.StandardDeviationPopulation);
|
||||
|
||||
var diagnostics = (CustomNodeManager2)nm.Server.DiagnosticsNodeManager;
|
||||
var folder = diagnostics.FindPredefinedNode(ObjectIds.HistoryServerCapabilities_AggregateFunctions, null)!;
|
||||
AggregateFunctionMap.ByNodeId.Keys.Count(id => folder.ReferenceExists(ReferenceTypeIds.Organizes, false, id))
|
||||
.ShouldBe(AggregateFunctionMap.ByNodeId.Count);
|
||||
|
||||
// Positive control on the same folder: a real (non-null) set trims it.
|
||||
nm.HistorianDataSource = NullHistorianDataSource.Instance;
|
||||
AggregateFunctionMap.ByNodeId.Keys.Count(id => folder.ReferenceExists(ReferenceTypeIds.Organizes, false, id))
|
||||
.ShouldBe(0);
|
||||
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #677: a client <c>AggregateConfiguration</c> that differs from the server defaults is refused per node with
|
||||
/// <c>BadAggregateConfigurationRejected</c> (it used to be silently ignored); the source is never invoked.
|
||||
@@ -674,8 +716,8 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
public bool ThrowOnRead { get; init; }
|
||||
public bool AggregateNotSupported { get; init; }
|
||||
public bool LimitExceeded { get; init; }
|
||||
public IReadOnlySet<HistoryAggregateType> Supported { get; init; } = HistoryAggregateTypes.All;
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => Supported;
|
||||
public IReadOnlySet<HistoryAggregateType>? Supported { get; init; } = HistoryAggregateTypes.All;
|
||||
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => Supported!;
|
||||
public HistorianRead RawResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
public HistorianRead ProcessedResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
public HistorianRead AtTimeResult { get; set; } = new(Array.Empty<DataValueSnapshot>(), null);
|
||||
|
||||
Reference in New Issue
Block a user