Merge master into feat/677-aggregates
# Conflicts: # docs/Historian.md
This commit is contained in:
@@ -1348,7 +1348,7 @@ historian tagname, which is the tag's driver `FullName`.
|
|||||||
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); Partial also marks the interval the data starts inside (not Average); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
|
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); Partial also marks the interval the data starts inside (not Average); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
|
||||||
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
||||||
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
|
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
|
||||||
- **HistoryUpdate** (SqlServer only; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`.
|
- **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization, same `IDriverPrimaryRoleSource` verdict as the native-ack gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ `BadHistoryOperationUnsupported` (nothing sent), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key).
|
||||||
- **Modified-value history (#668, Part 11):** every applied HistoryUpdate item leaves a trail row (prior value, principal from the session's API key / `anonymous`, `HistoryModificationType`, time) — SqlServer `hist.SampleModified` (**schema v2**, written by `OUTPUT … INTO` in the same statement ⇒ atomic; ⚠️ no FK/CHECK/trigger on it, and `AutoCreateSchema=false` installs must run `002_sample_modified.sql` before upgrading), QuestDB `{p}sample_modified` (append-only, written **before** the change ⇒ a crash leaves an orphan trail row, never an untracked change). `IsReadModified=true` HistoryRead serves it as `HistoryModifiedData` via `IHistorianDataSource.ReadModifiedAsync` (default ⇒ `NotSupported` ⇒ `BadHistoryOperationUnsupported` on Gateway/Null), paged on `(SourceTime, ModifiedId)`, gated by plain HistoryRead. Live writes and retention never write it; retention/TTL purge it on source time. `IHistorianDataUpdater` methods take a `HistoryUpdateContext`. CLI `historyread --modified`. `docs/Historian.md` §Modified-value history.
|
- **Modified-value history (#668, Part 11):** every applied HistoryUpdate item leaves a trail row (prior value, principal from the session's API key / `anonymous`, `HistoryModificationType`, time) — SqlServer `hist.SampleModified` (**schema v2**, written by `OUTPUT … INTO` in the same statement ⇒ atomic; ⚠️ no FK/CHECK/trigger on it, and `AutoCreateSchema=false` installs must run `002_sample_modified.sql` before upgrading), QuestDB `{p}sample_modified` (append-only, written **before** the change ⇒ a crash leaves an orphan trail row, never an untracked change). `IsReadModified=true` HistoryRead serves it as `HistoryModifiedData` via `IHistorianDataSource.ReadModifiedAsync` (default ⇒ `NotSupported` ⇒ `BadHistoryOperationUnsupported` on Gateway/Null), paged on `(SourceTime, ModifiedId)`, gated by plain HistoryRead. Live writes and retention never write it; retention/TTL purge it on source time. `IHistorianDataUpdater` methods take a `HistoryUpdateContext`. CLI `historyread --modified`. `docs/Historian.md` §Modified-value history.
|
||||||
|
|
||||||
### Read path (`ServerHistorian` section)
|
### Read path (`ServerHistorian` section)
|
||||||
|
|||||||
@@ -207,7 +207,9 @@ The `-v` drops the SQL volume; remove it to keep ConfigDb state across restarts.
|
|||||||
|
|
||||||
The full single-mesh stack (`central-1`/`central-2` + the four site nodes) can OOM-kill `central-1` on a loaded host. Two settings in the compose file guard against that:
|
The full single-mesh stack (`central-1`/`central-2` + the four site nodes) can OOM-kill `central-1` on a loaded host. Two settings in the compose file guard against that:
|
||||||
|
|
||||||
- **EF Core + ASP.NET Core logs are pinned to `Warning`** on every host node (`Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore` / `…Microsoft.AspNetCore` = `Warning`). The host logs via Serilog (`AddZbSerilog` → `ReadFrom.Configuration`), and in `Development` the default level is `Debug` — without these overrides every Deployment-poll emits an `Executed DbCommand` / `SELECT … FROM [Deployment]` line, flooding the Serilog pipeline and starving the Akka cluster heartbeat thread. Application + Akka log levels are left untouched, so this only silences the per-poll SQL chatter. To temporarily restore the SQL log flood for debugging, drop those two env vars (or set them back to `Information`) on the node you're inspecting.
|
- **EF Core + ASP.NET Core logs are pinned to `Warning`** on every host node (`Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore` / `…Microsoft.AspNetCore` = `Warning`). The host logs via Serilog (`AddZbSerilog` → `ReadFrom.Configuration`), and a `Development` environment would default to `Debug` (`appsettings.Development.json`) — the rig does NOT set `ASPNETCORE_ENVIRONMENT`, so nodes actually run at the base `Information` default, but these pins keep the SQL chatter out should the rig ever run as Development; without them a Debug/Information-level EF pipeline emits an `Executed DbCommand` / `SELECT … FROM [Deployment]` line, flooding the Serilog pipeline and starving the Akka cluster heartbeat thread. Application + Akka log levels are left untouched, so this only silences the per-poll SQL chatter. To temporarily restore the SQL log flood for debugging, drop those two env vars (or set them back to `Information`) on the node you're inspecting.
|
||||||
|
|
||||||
|
To raise Debug logging for chosen categories on `central-1`/`central-2` (e.g. `ZB.MOM.WW.OtOpcUa.Runtime`), use the overlay: `docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.debug.yml up -d --no-deps central-1 central-2` — it sets `Serilog__MinimumLevel__Override__<Namespace>=Debug`; edit its category list to taste.
|
||||||
- **Each host node has `mem_limit: 1g`** (`mem_reservation: 512m`). A quiet solo `central-1` measures ~357 MiB; the limit leaves headroom for the deploy/UI load and per-cluster driver subscriptions that push a fully-loaded node higher. The limit/reservation live on the `&otopcua-host` anchor, so all six host services inherit them; `sql`, `traefik`, and the one-shot `migrator`/`cluster-seed` are left unbounded.
|
- **Each host node has `mem_limit: 1g`** (`mem_reservation: 512m`). A quiet solo `central-1` measures ~357 MiB; the limit leaves headroom for the deploy/UI load and per-cluster driver subscriptions that push a fully-loaded node higher. The limit/reservation live on the `&otopcua-host` anchor, so all six host services inherit them; `sql`, `traefik`, and the one-shot `migrator`/`cluster-seed` are left unbounded.
|
||||||
|
|
||||||
The full six-node host stack therefore needs roughly **6 GiB** of Docker Desktop VM memory just for the host nodes (plus SQL Server's own footprint on top). On a constrained host, either raise the Docker Desktop VM memory or run fewer host services (e.g. just `central-1` + `central-2`, or a single central node) rather than the full mesh.
|
The full six-node host stack therefore needs roughly **6 GiB** of Docker Desktop VM memory just for the host nodes (plus SQL Server's own footprint on top). On a constrained host, either raise the Docker Desktop VM memory or run fewer host services (e.g. just `central-1` + `central-2`, or a single central node) rather than the full mesh.
|
||||||
|
|||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# docker-dev/docker-compose.debug.yml — optional overlay: raise Serilog levels per category on the central pair.
|
||||||
|
#
|
||||||
|
# The rig nodes run with ASPNETCORE_ENVIRONMENT unset (Production), so there is NO appsettings.Development.json and the
|
||||||
|
# Serilog default level is Information (the base appsettings.json sets none). Debug is therefore OFF by default; this
|
||||||
|
# overlay turns it on for chosen namespaces only, via Serilog:MinimumLevel:Override:<Namespace> (env: double underscore
|
||||||
|
# for ':', and the namespace's own dots stay dots).
|
||||||
|
#
|
||||||
|
# Up (recreates only the two central nodes with the extra env):
|
||||||
|
# docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.debug.yml up -d --no-deps central-1 central-2
|
||||||
|
# Back to normal: re-run `up -d --no-deps central-1 central-2` with the base file alone.
|
||||||
|
#
|
||||||
|
# Edit the list below to the categories you care about. Useful ones:
|
||||||
|
# ZB.MOM.WW.OtOpcUa.Runtime actors: DriverHostActor, OpcUaPublishActor, redundancy, deploys
|
||||||
|
# ZB.MOM.WW.OtOpcUa.Runtime.Drivers.DriverHostActor one actor only (e.g. the #672 historization verdict logs)
|
||||||
|
# ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient a single driver
|
||||||
|
# Opc.Ua OPC Foundation stack (very chatty)
|
||||||
|
# Akka cluster/remoting internals (chatty; base is Information)
|
||||||
|
# Leave Microsoft.EntityFrameworkCore / Microsoft.AspNetCore at Warning (see README "Resource limits & dev logging"):
|
||||||
|
# a Debug SQL flood starves the Akka heartbeat thread.
|
||||||
|
services:
|
||||||
|
central-1:
|
||||||
|
environment:
|
||||||
|
Serilog__MinimumLevel__Override__ZB.MOM.WW.OtOpcUa.Runtime: "Debug"
|
||||||
|
central-2:
|
||||||
|
environment:
|
||||||
|
Serilog__MinimumLevel__Override__ZB.MOM.WW.OtOpcUa.Runtime: "Debug"
|
||||||
+14
-3
@@ -57,6 +57,11 @@ otopcua-cli write -u opc.tcp://localhost:4840 -n "ns=2;s=MyNode" -v 42 -U operat
|
|||||||
|
|
||||||
### Failover
|
### Failover
|
||||||
|
|
||||||
|
**Mapped ports and internal hostnames.** Endpoint discovery rewrites the host *and port* of the endpoint the server
|
||||||
|
advertises to the URL you passed (when the scheme matches), so connecting through a port mapping works even though the
|
||||||
|
server advertises its internal address (for example the docker-dev rig: `-u opc.tcp://localhost:4841` reaches the
|
||||||
|
node mapped to 4841, although every container advertises `:4840`).
|
||||||
|
|
||||||
When `-F` is provided, the shared service tries the primary URL first, then each failover URL in order. For long-running commands (`subscribe`, `alarms`), the service monitors the session via keep-alive and automatically reconnects to the next available server on failure.
|
When `-F` is provided, the shared service tries the primary URL first, then each failover URL in order. For long-running commands (`subscribe`, `alarms`), the service monitors the session via keep-alive and automatically reconnects to the next available server on failure.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -229,7 +234,7 @@ continuation points up to `--max`. A server on the `Gateway` backend (no modific
|
|||||||
| `--start` | Start time in ISO 8601 UTC format, e.g. `2026-01-15T08:00:00Z` (default: 24 hours ago) |
|
| `--start` | Start time in ISO 8601 UTC format, e.g. `2026-01-15T08:00:00Z` (default: 24 hours ago) |
|
||||||
| `--end` | End time in ISO 8601 UTC format, e.g. `2026-01-15T09:00:00Z` (default: now) |
|
| `--end` | End time in ISO 8601 UTC format, e.g. `2026-01-15T09:00:00Z` (default: now) |
|
||||||
| `--max` | Maximum number of values (default: 1000) |
|
| `--max` | Maximum number of values (default: 1000) |
|
||||||
| `--aggregate` | Aggregate function: Average, Minimum, Maximum, Count, Start, End, StandardDeviation |
|
| `--aggregate` | Aggregate function: Average, Minimum, Maximum, Count, Start, End, StandardDeviation, TimeAverage, TimeAverage2, Total, Total2, Interpolative |
|
||||||
| `--interval` | Processing interval in milliseconds for aggregates (default: 3600000) |
|
| `--interval` | Processing interval in milliseconds for aggregates (default: 3600000) |
|
||||||
| `--modified` | Read modified-value history (Part 11 `IsReadModified`) instead of raw samples |
|
| `--modified` | Read modified-value history (Part 11 `IsReadModified`) instead of raw samples |
|
||||||
|
|
||||||
@@ -270,8 +275,9 @@ node-level code (`Good`, or `BadNoData` if the range held nothing). A node that
|
|||||||
updater, so no `HistoryWrite` access bit), answers `BadHistoryOperationUnsupported`; a `NaN`/`Infinity` double is `BadInvalidArgument`. The session's API key
|
updater, so no `HistoryWrite` access bit), answers `BadHistoryOperationUnsupported`; a `NaN`/`Infinity` double is `BadInvalidArgument`. The session's API key
|
||||||
(`-U apikey -P otua_<keyId>_<secret>`) needs the `HistoryUpdate` permission — via a `Cluster`-scoped grant when `Security:Acl:Enabled` is off, or a scoped
|
(`-U apikey -P otua_<keyId>_<secret>`) needs the `HistoryUpdate` permission — via a `Cluster`-scoped grant when `Security:Acl:Enabled` is off, or a scoped
|
||||||
grant on the node when it is on; a key with only `WriteOperate` gets `BadUserAccessDenied`. See [Historian.md §HistoryUpdate](Historian.md#historyupdate) and
|
grant on the node when it is on; a key with only `WriteOperate` gets `BadUserAccessDenied`. See [Historian.md §HistoryUpdate](Historian.md#historyupdate) and
|
||||||
[security.md](security.md#historyupdate-historian-writes). `historyread --aggregate` only knows the seven names in the table below; the SQL backend's other
|
[security.md](security.md#historyupdate-historian-writes). `historyread --aggregate` only knows the twelve names in the table below; the SQL backend's other
|
||||||
aggregates (Total, Range, Delta, PercentGood, …) need a different client.
|
aggregates (Range, Delta, PercentGood, …) need a different client. A refused history read (for example
|
||||||
|
`BadHistoryOperationUnsupported`) prints one error line and exits non-zero; it does not dump a stack trace.
|
||||||
|
|
||||||
`delete-raw` is sent with `IsDeleteModified=false`. Invalid or inconsistent options (unknown mode, unpaired
|
`delete-raw` is sent with `IsDeleteModified=false`. Invalid or inconsistent options (unknown mode, unpaired
|
||||||
value/time, unparsable value, options that do not apply to the mode) fail before any connection is made.
|
value/time, unparsable value, options that do not apply to the mode) fail before any connection is made.
|
||||||
@@ -287,6 +293,11 @@ value/time, unparsable value, options that do not apply to the mode) fail before
|
|||||||
| `Start` | `first` | `AggregateFunction_Start` |
|
| `Start` | `first` | `AggregateFunction_Start` |
|
||||||
| `End` | `last` | `AggregateFunction_End` |
|
| `End` | `last` | `AggregateFunction_End` |
|
||||||
| `StandardDeviation` | `stddev`, `stdev` | `AggregateFunction_StandardDeviationPopulation` |
|
| `StandardDeviation` | `stddev`, `stdev` | `AggregateFunction_StandardDeviationPopulation` |
|
||||||
|
| `TimeAverage` | `timeavg` | `AggregateFunction_TimeAverage` |
|
||||||
|
| `TimeAverage2` | `timeavg2` | `AggregateFunction_TimeAverage2` |
|
||||||
|
| `Total` | | `AggregateFunction_Total` |
|
||||||
|
| `Total2` | | `AggregateFunction_Total2` |
|
||||||
|
| `Interpolative` | `interp` | `AggregateFunction_Interpolative` |
|
||||||
|
|
||||||
### alarms
|
### alarms
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -206,7 +206,7 @@ All times are in UTC. Invalid input turns red on blur.
|
|||||||
|
|
||||||
| Option | Description |
|
| Option | Description |
|
||||||
|--------|-------------|
|
|--------|-------------|
|
||||||
| Aggregate | Raw (default), Average, Minimum, Maximum, Count, Start, End, Standard Deviation |
|
| Aggregate | Raw (default), Average, Minimum, Maximum, Count, Start, End, Standard Deviation, Time Average, Time Average 2, Total, Total 2, Interpolative |
|
||||||
| Interval (ms) | Processing interval for aggregate queries (shown only for aggregates) |
|
| Interval (ms) | Processing interval for aggregate queries (shown only for aggregates) |
|
||||||
| Max Values | Maximum number of raw values to return (default 1000) |
|
| Max Values | Maximum number of raw values to return (default 1000) |
|
||||||
|
|
||||||
|
|||||||
+26
-5
@@ -22,14 +22,30 @@ Two related concerns get their own dedicated pages and are **only summarised + l
|
|||||||
|
|
||||||
The Host (`Program.cs`) loads `appsettings.json`, then overlays a **per-role** file chosen from the cluster roles:
|
The Host (`Program.cs`) loads `appsettings.json`, then overlays a **per-role** file chosen from the cluster roles:
|
||||||
|
|
||||||
- A single role → `appsettings.{role}.json` (e.g. `appsettings.driver.json`, `appsettings.admin.json`).
|
- A single plain role → `appsettings.{role}.json` (e.g. `appsettings.driver.json`, `appsettings.admin.json`).
|
||||||
- Both roles → `appsettings.admin-driver.json` (roles joined with `-`, ordinal-sorted).
|
- Both roles → `appsettings.admin-driver.json` (plain roles joined with `-`, ordinal-sorted).
|
||||||
|
- `cluster-{ClusterId}` roles are **ignored** when naming the overlay (#678.6) — `[driver, cluster-SITE-A]` loads `appsettings.driver.json`.
|
||||||
- `appsettings.{ASPNETCORE_ENVIRONMENT}.json` (e.g. `appsettings.Development.json`) is layered on by the host builder.
|
- `appsettings.{ASPNETCORE_ENVIRONMENT}.json` (e.g. `appsettings.Development.json`) is layered on by the host builder.
|
||||||
|
|
||||||
All role overlays are **optional** — the base `appsettings.json` plus the Options-class C# defaults are enough to boot. The roles themselves come from `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`) — `Cluster:Roles`, falling back to `OTOPCUA_ROLES` when that is empty (see [`ServiceHosting.md`](ServiceHosting.md) and the table below).
|
All role overlays are **optional** — the base `appsettings.json` plus the Options-class C# defaults are enough to boot. The roles themselves come from `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`) — `Cluster:Roles`, falling back to `OTOPCUA_ROLES` when that is empty (see [`ServiceHosting.md`](ServiceHosting.md) and the table below).
|
||||||
|
|
||||||
The checked-in `appsettings*.json` files are deliberately thin: they carry only `Serilog` and the `Security:Ldap` overlay. Everything else (`OpcUa`, `Cluster`, `ConnectionStrings`/`ConfigDb`) binds from the Options-class defaults documented below unless an operator adds the section explicitly or supplies the corresponding environment variable.
|
The checked-in `appsettings*.json` files are deliberately thin: they carry only `Serilog` and the `Security:Ldap` overlay. Everything else (`OpcUa`, `Cluster`, `ConnectionStrings`/`ConfigDb`) binds from the Options-class defaults documented below unless an operator adds the section explicitly or supplies the corresponding environment variable.
|
||||||
|
|
||||||
|
<a id="role-overlay-audit"></a>
|
||||||
|
#### Role overlay audit (#678.6)
|
||||||
|
|
||||||
|
Until #678.6 the overlay never loaded on a node carrying a `cluster-*` role (the cluster role leaked into the filename). Fixing that **activates** the shipped overlays on those nodes (the docker-dev rig and any per-cluster-mesh production node), so each key was audited against base `appsettings.json` / the Options defaults. Env vars and command-line args are re-appended after the overlay, so any value a deployment sets explicitly (the rig sets every `Security__Ldap__*` key) still wins.
|
||||||
|
|
||||||
|
| Key | Overlay(s) | Base / default | Effect on activation |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `Serilog:MinimumLevel:Default` = `Information` | all three | Serilog default `Information` | no-op (in `Development` it outranks `appsettings.Development.json`'s `Debug`, by design) |
|
||||||
|
| `…Override:Akka` = `Information` | all three | inherits `Default` (`Information`) | no-op |
|
||||||
|
| `…Override:Microsoft.AspNetCore` = `Warning` | `admin`, `admin-driver` | unset (`Information`) | quieter ASP.NET Core logs; log-level only, matches what the rig already pins |
|
||||||
|
| `…Override:Opc.Ua` = `Information` | `admin-driver`, `driver` | inherits `Default` | no-op. **`driver` shipped `Debug` — changed to `Information` in #678.6**: activating a Debug OPC UA stack trace on every production driver node would flood Serilog (the same pipeline starvation the rig's EF/AspNetCore pins guard against). Raise it per node with `Serilog__MinimumLevel__Override__Opc.Ua=Debug`. |
|
||||||
|
| `Security:Ldap:DevStubMode` = `false` | all three | `false` | no-op; the overlay's point is that it outranks a developer's `appsettings.Development.json` (`true`) |
|
||||||
|
| `Security:Ldap:Transport` = `Ldaps` | all three | `LdapOptions.Transport` default `None` | **behaviour-changing, kept deliberately** (documented secure production default). A cluster-role node that configures `Security:Ldap:Server`/`Port` for a plaintext LDAP but never sets `Transport` would now dial LDAPS and fail AdminUI sign-in; set `Security__Ldap__Transport` (the rig already sets `None` explicitly). Data-plane auth is API-key based and unaffected; driver-only nodes have no AdminUI. |
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `appsettings` sections
|
## `appsettings` sections
|
||||||
@@ -38,7 +54,7 @@ The checked-in `appsettings*.json` files are deliberately thin: they carry only
|
|||||||
|
|
||||||
- **Purpose:** logging. Console + rolling daily file sink, layered with the shared `ZB.MOM.WW.Telemetry` enrichers (`AddZbSerilog` in `Program.cs`).
|
- **Purpose:** logging. Console + rolling daily file sink, layered with the shared `ZB.MOM.WW.Telemetry` enrichers (`AddZbSerilog` in `Program.cs`).
|
||||||
- **Where bound:** `builder.AddZbSerilog(...)` reads `Serilog` from configuration (`ReadFrom.Configuration`).
|
- **Where bound:** `builder.AddZbSerilog(...)` reads `Serilog` from configuration (`ReadFrom.Configuration`).
|
||||||
- **Checked-in shape** (`appsettings.json`): `Using` = `[ "Serilog.Sinks.Console", "Serilog.Sinks.File" ]`, `WriteTo` = a `Console` sink and a `File` sink (`path: logs/otopcua-.log`, `rollingInterval: Day`). Role overlays add `MinimumLevel` / `Override` blocks (e.g. `Opc.Ua: Debug`, `Akka: Information`).
|
- **Checked-in shape** (`appsettings.json`): `Using` = `[ "Serilog.Sinks.Console", "Serilog.Sinks.File" ]`, `WriteTo` = a `Console` sink and a `File` sink (`path: logs/otopcua-.log`, `rollingInterval: Day`). Role overlays add `MinimumLevel` / `Override` blocks (e.g. `Opc.Ua: Information`, `Akka: Information`).
|
||||||
|
|
||||||
### `OpcUa`
|
### `OpcUa`
|
||||||
|
|
||||||
@@ -410,7 +426,7 @@ Validated at host start (driver-role nodes) when `ServerHistorian:Enabled` **or*
|
|||||||
| `ServerHistorian:SqlServer:ConnectionString` | `""` | **Required.** Historian database connection string (not the ConfigDb). Supply via `${secret:OT-HISTORIAN-SQL}` or env `ServerHistorian__SqlServer__ConnectionString`; the database must already exist. |
|
| `ServerHistorian:SqlServer:ConnectionString` | `""` | **Required.** Historian database connection string (not the ConfigDb). Supply via `${secret:OT-HISTORIAN-SQL}` or env `ServerHistorian__SqlServer__ConnectionString`; the database must already exist. |
|
||||||
| `ServerHistorian:SqlServer:Schema` | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (spliced into DDL/DML). |
|
| `ServerHistorian:SqlServer:Schema` | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (spliced into DDL/DML). |
|
||||||
| `ServerHistorian:SqlServer:CommandTimeout` | `00:00:30` | Must be `> 0`; rounded up to whole seconds. |
|
| `ServerHistorian:SqlServer:CommandTimeout` | `00:00:30` | Must be `> 0`; rounded up to whole seconds. |
|
||||||
| `ServerHistorian:SqlServer:AutoCreateSchema` | `true` | `true` applies the embedded schema script under an app-lock on first use; `false` only verifies `SchemaVersion >= 1` and fails every historian call (health `LastError`, no crash) if behind. |
|
| `ServerHistorian:SqlServer:AutoCreateSchema` | `true` | `true` applies the embedded schema scripts under an app-lock on first use (upgrading a v1 schema to v2); `false` only verifies `SchemaVersion >= 1` and fails every historian call (health `LastError`, no crash) if behind. A verified **v1** schema (no `002_sample_modified.sql`) is served with modified-value reads and HistoryUpdate refused `BadHistoryOperationUnsupported`, retention skipping `SampleModified`, and one Warning per boot (#674). |
|
||||||
| `ServerHistorian:SqlServer:RetentionDays` | unset | Days of history kept (source time), `>= 1` when set; unset keeps everything. Enables the Primary-gated purge of `hist.Sample`/`hist.Event` (see [Historian.md](Historian.md#retention-retentiondays)). |
|
| `ServerHistorian:SqlServer:RetentionDays` | unset | Days of history kept (source time), `>= 1` when set; unset keeps everything. Enables the Primary-gated purge of `hist.Sample`/`hist.Event` (see [Historian.md](Historian.md#retention-retentiondays)). |
|
||||||
| `ServerHistorian:SqlServer:PurgeInterval` | `01:00:00` | Purge cadence; must be `> 0`. |
|
| `ServerHistorian:SqlServer:PurgeInterval` | `01:00:00` | Purge cadence; must be `> 0`. |
|
||||||
| `ServerHistorian:SqlServer:PurgeBatchSize` | `4000` | Rows per `DELETE TOP (n)` batch; `>= 1`. Keep under ~5000 to avoid lock escalation. |
|
| `ServerHistorian:SqlServer:PurgeBatchSize` | `4000` | Rows per `DELETE TOP (n)` batch; `>= 1`. Keep under ~5000 to avoid lock escalation. |
|
||||||
@@ -430,7 +446,12 @@ when `ServerHistorian:Enabled` **or** `AlarmHistorian:Enabled` is true (`QuestDb
|
|||||||
| `ServerHistorian:QuestDb:VisibilityTimeout` | `00:00:05` | Must be `>= 0`. How long a write waits for its rows to become visible to queries (WAL apply); `0` disables the wait. A timeout only logs a Warning. |
|
| `ServerHistorian:QuestDb:VisibilityTimeout` | `00:00:05` | Must be `>= 0`. How long a write waits for its rows to become visible to queries (WAL apply); `0` disables the wait. A timeout only logs a Warning. |
|
||||||
|
|
||||||
HistoryUpdate (SqlServer and QuestDb only; QuestDb deletes are soft — tombstones) additionally needs `ServerHistorian:Enabled=true` and a key
|
HistoryUpdate (SqlServer and QuestDb only; QuestDb deletes are soft — tombstones) additionally needs `ServerHistorian:Enabled=true` and a key
|
||||||
whose principal holds the `HistoryUpdate` permission — see [`docs/security.md`](security.md#historyupdate-historian-writes). Full semantics,
|
whose principal holds the `HistoryUpdate` permission — see [`docs/security.md`](security.md#historyupdate-historian-writes). It is served
|
||||||
|
only on the redundancy **Primary** (`BadInvalidState` elsewhere, #674) and is bounded by one backend-independent key:
|
||||||
|
|
||||||
|
| Key | Default | Meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `ServerHistorian:HistoryUpdateMaxFutureSkew` | `1.00:00:00` (1 day) | TimeSpan. A HistoryUpdate (Insert / Replace / Update) item whose `SourceTimestamp` is later than server-clock now + this is refused `BadOutOfRange` before any backend sees it (#674 — a far-future typo could otherwise age out a QuestDB TTL table or poison the history). Applies to every backend; the live recorder is not bounded. Non-positive disables the bound. | Full semantics,
|
||||||
schema, aggregates and the docker-dev recipe: [`docs/Historian.md`](Historian.md#sql-server-backend) (SqlServer) and
|
schema, aggregates and the docker-dev recipe: [`docs/Historian.md`](Historian.md#sql-server-backend) (SqlServer) and
|
||||||
[`docs/Historian.md`](Historian.md#questdb-backend) (QuestDb: the µs timestamp rules, pre-1970 limits, TTL and the docker-dev overlay).
|
[`docs/Historian.md`](Historian.md#questdb-backend) (QuestDb: the µs timestamp rules, pre-1970 limits, TTL and the docker-dev overlay).
|
||||||
|
|
||||||
|
|||||||
+86
-19
@@ -85,7 +85,8 @@ consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `A
|
|||||||
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` | `ServerHistorian:QuestDb:*` |
|
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` | `ServerHistorian:QuestDb:*` |
|
||||||
|
|
||||||
Recording stays **Primary-only** (#537) and **at-least-once** with the #538 per-tag quarantine on every backend;
|
Recording stays **Primary-only** (#537) and **at-least-once** with the #538 per-tag quarantine on every backend;
|
||||||
HistoryRead and HistoryUpdate are served from any node (the database is shared). The historian connection string is
|
HistoryRead is served from any node (the database is shared). HistoryUpdate is **Primary-only** since #674
|
||||||
|
(`BadInvalidState` on any other node — see [HistoryUpdate](#primary-only-674)). The historian connection string is
|
||||||
**not** a ConfigDb string, and a driver-only node may carry it without breaking the Phase 4 "no ConfigDb" posture.
|
**not** a ConfigDb string, and a driver-only node may carry it without breaking the Phase 4 "no ConfigDb" posture.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -145,6 +146,7 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
|
|||||||
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` only (#671). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped. 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). |
|
||||||
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||||
|
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
|
||||||
|
|
||||||
**`ServerHistorian:SqlServer`** (read only when `Backend=SqlServer`; bound by `HistorianBackendRegistry.BindSqlServer`):
|
**`ServerHistorian:SqlServer`** (read only when `Backend=SqlServer`; bound by `HistorianBackendRegistry.BindSqlServer`):
|
||||||
|
|
||||||
@@ -630,13 +632,31 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
|||||||
|
|
||||||
- The cursor is `(source, next start, end, boundary skip, cap)` in the same session-bound store as Raw (flagged as an events
|
- The cursor is `(source, next start, end, boundary skip, cap)` in the same session-bound store as Raw (flagged as an events
|
||||||
cursor — a Raw point presented to an events read, or the reverse, is `BadContinuationPointInvalid`). The next page reads from
|
cursor — a Raw point presented to an events read, or the reverse, is `BadContinuationPointInvalid`). The next page reads from
|
||||||
the last event's `EventTimeUtc` **inclusive** and drops the events at that timestamp already emitted.
|
the last event's `EventTimeUtc` **inclusive** and drops the events at that timestamp already emitted — and asks the source for
|
||||||
|
that many more rows than the cap, so every page short of the last is a FULL `NumValuesPerNode` page (#675; resumed pages used
|
||||||
|
to come back short by the skip).
|
||||||
- A tie cluster larger than the cap is paged within by over-fetching the half-open window `[T, T + resolution)` up to
|
- A tie cluster larger than the cap is paged within by over-fetching the half-open window `[T, T + resolution)` up to
|
||||||
`MaxTieClusterOverfetch`; beyond that bound the node answers `BadHistoryOperationUnsupported` (raise the bound).
|
`MaxTieClusterOverfetch`; beyond that bound the node answers `BadHistoryOperationUnsupported` (raise the bound).
|
||||||
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
|
- **Tie exactness depends on the backend.** SqlServer and QuestDB order by `(EventTimeUtc, EventId)`, so the skip count is exact.
|
||||||
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary are
|
The Gateway backend has no stable intra-timestamp order (and only a truncation flag), so ties at a page boundary are
|
||||||
best-effort there.
|
best-effort there.
|
||||||
- Reverse event reads (`start > end`) are not paged server-side; no continuation point is issued for them.
|
- **Reverse event reads page newest-first (#675).** `StartTime > EndTime` reads the Part 11 reverse window `(EndTime, StartTime]`
|
||||||
|
with events in DESCENDING `(EventTimeUtc, EventId)` order, so a cap keeps the **latest** N and a continuation point is issued
|
||||||
|
when the source flags more. The direction is stored **in the cursor**: a reverse point resumes reverse (and a forward one
|
||||||
|
forward) whatever the resume request's details say. Reverse tie clusters are over-fetched forward and sliced from the newest
|
||||||
|
end, exactly as Raw does. Before #675 a capped reverse read silently returned the EARLIEST N, ascending, Good, with no
|
||||||
|
continuation point.
|
||||||
|
- **End-only events reads read backward (#675).** `StartTime` unset (`DateTime.MinValue`) with an `EndTime` and
|
||||||
|
`NumValuesPerNode > 0` pages backward from `EndTime` over `(MinValue, EndTime]` — the same rule as Raw. Start-only reads still
|
||||||
|
page forward to an open upper bound (`DateTime.MaxValue`, UTC). Without a cap an end-only read keeps its forward
|
||||||
|
`[MinValue, EndTime)` meaning.
|
||||||
|
- **Backends.** The server calls `IHistorianDataSource.ReadEventsAsync(…, bool reverse, …)`. SqlServer and QuestDB serve reverse
|
||||||
|
natively (`… > lo AND … <= hi ORDER BY time DESC, id DESC`). **Gateway and Null refuse a reverse read** (the interface default
|
||||||
|
throws `NotSupportedException`) and the node answers `BadHistoryOperationUnsupported` — the gateway streams ascending only and
|
||||||
|
rejects `end < start`, and draining the whole window to keep its tail would be exactly the unbounded read the cap prevents.
|
||||||
|
- **Gateway open upper bound.** The `DateTime.MaxValue` (UTC) bound of a start-only forward read is pinned offline through the
|
||||||
|
published client's proto `Timestamp` conversion (captured by an in-process gRPC server) and the FILETIME encoding
|
||||||
|
(`GatewayEventBoundsTests`). What a live AVEVA Historian does with a year-9999 bound is still unverified (needs the VPN).
|
||||||
- No session on the request (in-process callers) degrades to single-shot, as for Raw.
|
- No session on the request (in-process callers) degrades to single-shot, as for Raw.
|
||||||
|
|
||||||
### Modified-value history (Part 11)
|
### Modified-value history (Part 11)
|
||||||
@@ -686,6 +706,25 @@ modification time and the Part 11 `HistoryUpdateType`. A HistoryReadRaw with `Is
|
|||||||
within one node the order is strict.
|
within one node the order is strict.
|
||||||
- **Retention.** The trail ages out on the same **source-time** cutoff as the samples it describes: `SqlServer` purges `hist.SampleModified`
|
- **Retention.** The trail ages out on the same **source-time** cutoff as the samples it describes: `SqlServer` purges `hist.SampleModified`
|
||||||
per tag alongside `hist.Sample`; QuestDB puts the same TTL on `{p}sample_modified`.
|
per tag alongside `hist.Sample`; QuestDB puts the same TTL on `{p}sample_modified`.
|
||||||
|
- **SqlServer schema v1 (#674).** With `AutoCreateSchema=false` a database still at schema **version 1** (script `002_sample_modified.sql` not
|
||||||
|
applied) keeps historizing — live writes, raw/processed/at-time reads, alarm events and retention all work — but has no trail: modified reads
|
||||||
|
and every HistoryUpdate item answer **`BadHistoryOperationUnsupported`** with nothing sent to SQL, retention skips `SampleModified`, and the
|
||||||
|
node logs **one Warning per boot**. Apply `002_sample_modified.sql` and restart the node to enable them (the verdict is taken once per
|
||||||
|
process). `AutoCreateSchema=true` (the default) still upgrades v1 to v2 in place. See [Schema](#schema).
|
||||||
|
- **QuestDB retry duplicates (#674).** `{p}sample_modified` is append-only with **no DEDUP key** (a key would collapse genuinely distinct
|
||||||
|
modifications of one sample), and each trail row gets a fresh `modified_id`. So a HistoryUpdate a client **retries** after a timeout — when the
|
||||||
|
first attempt had in fact landed — records a second trail row for the same change (and, for Insert, the retry is `BadEntryExists`). The
|
||||||
|
sample itself is idempotent (dedup upsert); only the audit trail can show the duplicate. `SqlServer` has the same exposure only if the client
|
||||||
|
retries a committed request; there is no request id to dedupe on in either backend.
|
||||||
|
- **Cross-node ordering (QuestDB).** The per-tag lock is per process; the [Primary-only](#primary-only-674) HistoryUpdate gate (#674) is what keeps
|
||||||
|
two nodes of a pair from recording concurrent updates of one tag (and so a wrong prior value). Two **separate** pairs sharing one QuestDB can
|
||||||
|
still race; that is unsupported.
|
||||||
|
- ⚠️ **Who can read the trail (#674.4).** Modified reads are ordinary HistoryReads: with `Security:Acl:Enabled=false` (the default) **any admitted
|
||||||
|
session, including Anonymous, can read who rewrote history** (`UserName` is the principal). Turn the ACL flag on, or set
|
||||||
|
`OpcUa:AllowAnonymous=false`, if the principal names are sensitive (see [security.md](security.md)).
|
||||||
|
- **Not implemented (#674.5):** `IsDeleteModified` (deleting trail entries — `BadHistoryOperationUnsupported`); recording an **Update** (upsert) as
|
||||||
|
a split `Insert` / `Replace` — it is recorded as `Update` either way; and **`AuditHistoryUpdateEvent`** (no audit event is emitted for a history
|
||||||
|
change — the trail is the only record).
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
otopcua-cli historyread -u opc.tcp://localhost:4840 -U apikey -P otua_<keyId>_<secret> \
|
otopcua-cli historyread -u opc.tcp://localhost:4840 -U apikey -P otua_<keyId>_<secret> \
|
||||||
@@ -758,7 +797,7 @@ The bootstrapper applies every script above the recorded version, in order, unde
|
|||||||
|
|
||||||
| Object | Purpose |
|
| Object | Purpose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `SchemaVersion(Version, AppliedUtc)` | Applied-script ledger. Required version: **2** (since #668). |
|
| `SchemaVersion(Version, AppliedUtc)` | Applied-script ledger. Required version: **1**; version **2** adds the modification trail (#668). A v1 database is served with the trail features off (#674 — see below). |
|
||||||
| `Tag(TagId int identity, Name nvarchar(400) unique, DataType tinyint, EngineeringUnit, Description, CreatedUtc, UpdatedUtc)` | One row per historian tagname. `DataType` is a `DriverDataType`. |
|
| `Tag(TagId int identity, Name nvarchar(400) unique, DataType tinyint, EngineeringUnit, Description, CreatedUtc, UpdatedUtc)` | One row per historian tagname. `DataType` is a `DriverDataType`. |
|
||||||
| `Sample(TagId, SourceTimeUtc datetime2(7), ValueKind tinyint, NumValue float, IntValue bigint, StrValue nvarchar(max), StatusCode int, ServerTimeUtc)` | **PK `(TagId, SourceTimeUtc)`** — one value per timestamp (Part 11), which makes redelivery idempotent and rules out tie clusters. `StatusCode` is the uint status stored bit-for-bit. |
|
| `Sample(TagId, SourceTimeUtc datetime2(7), ValueKind tinyint, NumValue float, IntValue bigint, StrValue nvarchar(max), StatusCode int, ServerTimeUtc)` | **PK `(TagId, SourceTimeUtc)`** — one value per timestamp (Part 11), which makes redelivery idempotent and rules out tie clusters. `StatusCode` is the uint status stored bit-for-bit. |
|
||||||
| `Event(EventId nvarchar(64) PK nonclustered, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, Message, UserName, Comment, EventTimeUtc, ReceivedTimeUtc)` | Alarm history. Clustered on `(EventTimeUtc, EventId)`; extra index `(SourceName, EventTimeUtc)`. |
|
| `Event(EventId nvarchar(64) PK nonclustered, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, Message, UserName, Comment, EventTimeUtc, ReceivedTimeUtc)` | Alarm history. Clustered on `(EventTimeUtc, EventId)`; extra index `(SourceName, EventTimeUtc)`. |
|
||||||
@@ -774,11 +813,14 @@ everywhere (full 100 ns precision — never a `DataTable`, which would round to
|
|||||||
`sqlcmd -S <server> -U <user> -P <pw> -d <db> -I -v Schema=hist -i docs/historian/sqlserver/001_initial.sql`, then the same with
|
`sqlcmd -S <server> -U <user> -P <pw> -d <db> -I -v Schema=hist -i docs/historian/sqlserver/001_initial.sql`, then the same with
|
||||||
`002_sample_modified.sql` (`-v Schema=` supplies the `$(Schema)` placeholder; verified against the docker-dev SQL container). Re-running is a no-op.
|
`002_sample_modified.sql` (`-v Schema=` supplies the `$(Schema)` placeholder; verified against the docker-dev SQL container). Re-running is a no-op.
|
||||||
|
|
||||||
⚠️ **Upgrading an existing v1 historian (#668).** A build with this change requires schema **version 2**. With `AutoCreateSchema=true`
|
⚠️ **Upgrading an existing v1 historian (#668, #674).** With `AutoCreateSchema=true` (the default) the first historian call after the
|
||||||
(the default) the first historian call after the upgrade applies `002_sample_modified.sql` in place (it only creates `SampleModified`; existing
|
upgrade applies `002_sample_modified.sql` in place (it only creates `SampleModified`; existing samples are untouched). With
|
||||||
samples are untouched). With `AutoCreateSchema=false` **every historian call fails** (`missing version 2 (found 1)`, in health `LastError`)
|
`AutoCreateSchema=false` a **v1 database is accepted** (#674 — between #668 and #674 it refused every historian call, continuous-historization
|
||||||
until an operator runs `002_sample_modified.sql` by hand as above — do that before rolling the new build out. Modifications made before the
|
drains included): writes, reads, events and retention work, while modified-value reads and HistoryUpdate report
|
||||||
upgrade have no trail; the trail starts with the first HistoryUpdate after it.
|
`BadHistoryOperationUnsupported` without touching SQL, retention skips `SampleModified`, and one Warning per boot names the missing script.
|
||||||
|
Run `002_sample_modified.sql` by hand as above and **restart the node** to enable them (the version is read once per process). Only a schema
|
||||||
|
below version 1 (`missing version 1 (found 0)`) still fails every call. Modifications made before the upgrade have no trail; the trail starts
|
||||||
|
with the first HistoryUpdate after it.
|
||||||
|
|
||||||
### Writes
|
### Writes
|
||||||
|
|
||||||
@@ -843,8 +885,10 @@ result never has both.
|
|||||||
|
|
||||||
### Events
|
### Events
|
||||||
|
|
||||||
`[start, end)` by `EventTimeUtc`; **a reversed window (`end < start`) is swapped and returned ascending**. A null `sourceName` means all
|
`[start, end)` by `EventTimeUtc`; through the five-argument read **a reversed window (`end < start`) is swapped and returned
|
||||||
sources, otherwise `SourceName = @source`. Ordered `(EventTimeUtc, EventId)` ascending. A bounded read fetches `maxEvents + 1` rows and
|
ascending**. The direction-aware overload with `reverse: true` (what the node manager calls for a reverse read, #675) reads
|
||||||
|
`(lo, hi]` ordered `(EventTimeUtc, EventId)` **descending** — the clustered index scanned backwards. A null `sourceName` means all
|
||||||
|
sources, otherwise `SourceName = @source`. Forward is ordered `(EventTimeUtc, EventId)` ascending. A bounded read fetches `maxEvents + 1` rows and
|
||||||
returns a non-null continuation point when the extra row exists; the node manager turns that flag into an OPC UA continuation
|
returns a non-null continuation point when the extra row exists; the node manager turns that flag into an OPC UA continuation
|
||||||
point (see [Continuation-point paging (Events)](#continuation-point-paging-events-669)).
|
point (see [Continuation-point paging (Events)](#continuation-point-paging-events-669)).
|
||||||
|
|
||||||
@@ -1042,6 +1086,8 @@ equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the
|
|||||||
timeout 0) lets one purger run and the others skip that tick (Debug log).
|
timeout 0) lets one purger run and the others skip that tick (Debug log).
|
||||||
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|SampleModified|Event}`; a failed pass logs a Warning and retries next
|
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|SampleModified|Event}`; a failed pass logs a Warning and retries next
|
||||||
interval — it never affects capture or reads.
|
interval — it never affects capture or reads.
|
||||||
|
- **Schema v1 (#674).** On a v1 schema (no `SampleModified`, `AutoCreateSchema=false`) the trail purge is skipped and omitted from the
|
||||||
|
pass results; samples and events purge normally.
|
||||||
- **Not done: partition switching.** For very large installs (billions of rows) a partitioned `hist.Sample` with sliding-window
|
- **Not done: partition switching.** For very large installs (billions of rows) a partitioned `hist.Sample` with sliding-window
|
||||||
`SWITCH`/drop is cheaper than row deletes; the purger is deliberately simple batched deletes. Another table is added by one more
|
`SWITCH`/drop is cheaper than row deletes; the purger is deliberately simple batched deletes. Another table is added by one more
|
||||||
`DeleteLoopAsync` call in `SqlServerHistorianRetention.PurgeAsync` (as `hist.SampleModified` was, #668).
|
`DeleteLoopAsync` call in `SqlServerHistorianRetention.PurgeAsync` (as `hist.SampleModified` was, #668).
|
||||||
@@ -1222,11 +1268,12 @@ Every query filters `NOT deleted`; every value goes through `StoredSample`, so t
|
|||||||
from the last timestamp ± `TimestampResolutionTicks`.
|
from the last timestamp ± `TimestampResolutionTicks`.
|
||||||
- **AtTime:** previous-or-equal and next neighbour of each time by `LIMIT 1` seeks, `UNION ALL`-ed, **50 times per statement** (QuestDB 10.0.1 refuses
|
- **AtTime:** previous-or-equal and next neighbour of each time by `LIMIT 1` seeks, `UNION ALL`-ed, **50 times per statement** (QuestDB 10.0.1 refuses
|
||||||
more than 126 branches). `AtTimeEvaluator` decides each value, as on `SqlServer`.
|
more than 126 branches). `AtTimeEvaluator` decides each value, as on `SqlServer`.
|
||||||
- **Events:** `[start, end)` by `ts`, ordered `(ts, event_id)`, `maxEvents + 1` fetched to produce the continuation point.
|
- **Events:** `[start, end)` by `ts`, ordered `(ts, event_id)`, `maxEvents + 1` fetched to produce the continuation point; a
|
||||||
|
reverse read (#675) is `(lo, hi]` ordered `(ts, event_id)` descending.
|
||||||
- **Processed:** one query per node; bucket index `(cast(ts AS LONG) * 10 - bucketBaseUnixTicks) / intervalTicks` (never `SAMPLE BY`, whose alignment
|
- **Processed:** one query per node; bucket index `(cast(ts AS LONG) * 10 - bucketBaseUnixTicks) / intervalTicks` (never `SAMPLE BY`, whose alignment
|
||||||
differs), so bucket edges are identical to `SqlServer` (forward start-inclusive / end-exclusive with the partial bucket last; reverse anchored at the
|
differs), so bucket edges are identical to `SqlServer` (forward start-inclusive / end-exclusive with the partial bucket last; reverse anchored at the
|
||||||
later time, later-edge-inclusive, partial bucket earliest — #677; interval 0 ⇒ one bucket, 100,000-bucket cap). A second `LIMIT 1` statement fetches
|
later time, later-edge-inclusive, partial bucket earliest — #677; interval 0 ⇒ one bucket, 100,000-bucket cap). On a forward read of any aggregate
|
||||||
the tag's earliest live sample for the start-of-data Partial bit.
|
but Average, a second `LIMIT 1` statement (capped at the window end) fetches the tag's earliest live sample for the start-of-data Partial bit.
|
||||||
Min/max times use `row_number()` windows with the earliest-timestamp tie-break.
|
Min/max times use `row_number()` windows with the earliest-timestamp tie-break.
|
||||||
|
|
||||||
### Retention (TTL)
|
### Retention (TTL)
|
||||||
@@ -1240,7 +1287,14 @@ expired day partitions itself, tombstones included.
|
|||||||
re-read to verify; a very large `RetentionDays` that QuestDB normalises into MONTH/YEAR is re-`ALTER`ed on every boot (harmless).
|
re-read to verify; a very large `RetentionDays` that QuestDB normalises into MONTH/YEAR is re-`ALTER`ed on every boot (harmless).
|
||||||
- ⚠️ **TTL is enforced only on write commits.** QuestDB checks for expired partitions when a write commits to the table, so **a quiet table never
|
- ⚠️ **TTL is enforced only on write commits.** QuestDB checks for expired partitions when a write commits to the table, so **a quiet table never
|
||||||
drops anything**: partitions older than the TTL linger until the next write lands. Do not read a stale-looking partition as a broken TTL.
|
drops anything**: partitions older than the TTL linger until the next write lands. Do not read a stale-looking partition as a broken TTL.
|
||||||
- Expiry is judged against the wall clock, so a future-dated sample does not expire current data.
|
- **Expiry is judged against the wall clock** (`min(max timestamp, now)`), so a future-dated sample does not expire current data.
|
||||||
|
⚠️ **Keep `cairo.ttl.use.wall.clock` at its default, `true`** (#674). It is what QuestDB documents as protecting a TTL table from a
|
||||||
|
future-dated row; with it `false` the TTL would be measured from the table's newest timestamp and one far-future row (a 2062 typo) could
|
||||||
|
drop every older partition — including `{p}sample_modified`, the record of who changed history. Evidence (QuestDB 10.0.1, 2026-10-11): the
|
||||||
|
default is `true`, and a 2066 row inserted into a 30-day-TTL table evicted nothing inside the TTL, while a partition genuinely older than
|
||||||
|
30 days was evicted by that same commit — pinned by `TtlWallClockTests`. (The same run with the flag `false` also kept the in-TTL
|
||||||
|
partitions, so the test pins the outcome rather than isolating the flag; do not rely on that.) The server-side
|
||||||
|
`ServerHistorian:HistoryUpdateMaxFutureSkew` bound (default 1 day) is the primary guard: a HistoryUpdate cannot write a far-future row.
|
||||||
- The TTL is visible in table metadata: `SELECT table_name, ttlValue, ttlUnit FROM tables() WHERE starts_with(table_name, 'hist_')`.
|
- The TTL is visible in table metadata: `SELECT table_name, ttlValue, ttlUnit FROM tables() WHERE starts_with(table_name, 'hist_')`.
|
||||||
|
|
||||||
### Docker-dev recipe
|
### Docker-dev recipe
|
||||||
@@ -1281,8 +1335,8 @@ dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Integration
|
|||||||
|
|
||||||
OPC UA **HistoryUpdate** lets a client insert, replace, update and delete stored raw history. It is available only when the selected
|
OPC UA **HistoryUpdate** lets a client insert, replace, update and delete stored raw history. It is available only when the selected
|
||||||
backend provides an updater — **`SqlServer` and `QuestDb` do; `Gateway` does not**, and an updater is registered only when `ServerHistorian:Enabled=true`.
|
backend provides an updater — **`SqlServer` and `QuestDb` do; `Gateway` does not**, and an updater is registered only when `ServerHistorian:Enabled=true`.
|
||||||
Updates go straight to the backend (not through the outbox), are **not Primary-gated** (the database is shared), and share the HistoryRead
|
Updates go straight to the backend (not through the outbox), are **Primary-only** (#674, [below](#primary-only-674)), and share the
|
||||||
batch limiter and `HistoryReadDeadline`.
|
HistoryRead batch limiter and `HistoryReadDeadline`.
|
||||||
|
|
||||||
### Operations
|
### Operations
|
||||||
|
|
||||||
@@ -1300,7 +1354,8 @@ caller's principal, in the [modification trail](#modified-value-history-part-11)
|
|||||||
mixed (Part 11 reports item outcomes in `OperationResults`); it is Bad only for whole-node failures.
|
mixed (Part 11 reports item outcomes in `OperationResults`); it is Bad only for whole-node failures.
|
||||||
|
|
||||||
**Per-item rejections** (answered without touching SQL; other items still proceed): a null item ⇒ `BadInvalidArgument`;
|
**Per-item rejections** (answered without touching SQL; other items still proceed): a null item ⇒ `BadInvalidArgument`;
|
||||||
`SourceTimestamp == DateTime.MinValue` ⇒ `BadInvalidTimestamp`; a value the historian cannot type (array, unsupported CLR type) ⇒
|
`SourceTimestamp == DateTime.MinValue` ⇒ `BadInvalidTimestamp`; a **`SourceTimestamp` later than `now + ServerHistorian:HistoryUpdateMaxFutureSkew`**
|
||||||
|
(default 1 day, server clock; #674) ⇒ **`BadOutOfRange`** — on every backend, never sent; a value the historian cannot type (array, unsupported CLR type) ⇒
|
||||||
`BadTypeMismatch`; a **`NaN` / `±Infinity` `Double` ⇒ `BadInvalidArgument`** — it cannot live in a SQL `float`, and unlike the live-value
|
`BadTypeMismatch`; a **`NaN` / `±Infinity` `Double` ⇒ `BadInvalidArgument`** — it cannot live in a SQL `float`, and unlike the live-value
|
||||||
writer (which stores it as `Empty`) an operator's explicit update is rejected rather than silently rewritten; and a **null value with a
|
writer (which stores it as `Empty`) an operator's explicit update is rejected rather than silently rewritten; and a **null value with a
|
||||||
Good or Uncertain status ⇒ `BadInvalidArgument`** — an `Empty` sample is always Bad, so a null value is accepted only with a Bad status
|
Good or Uncertain status ⇒ `BadInvalidArgument`** — an `Empty` sample is always Bad, so a null value is accepted only with a Bad status
|
||||||
@@ -1323,7 +1378,19 @@ outcome, later ones `BadNoEntryExists`. **Unknown tag:** Insert / Update create
|
|||||||
`ServerHistorian:Enabled=false`); `PerformUpdateType.Remove`; `DeleteRawModifiedDetails.IsDeleteModified=true` (the
|
`ServerHistorian:Enabled=false`); `PerformUpdateType.Remove`; `DeleteRawModifiedDetails.IsDeleteModified=true` (the
|
||||||
[modification trail](#modified-value-history-part-11) is an audit record, aged out only by retention); and the out-of-scope detail types — **event** HistoryUpdate (`UpdateEventDetails` / `DeleteEventDetails`) and `UpdateStructureData` —
|
[modification trail](#modified-value-history-part-11) is an audit record, aged out only by retention); and the out-of-scope detail types — **event** HistoryUpdate (`UpdateEventDetails` / `DeleteEventDetails`) and `UpdateStructureData` —
|
||||||
which are not overridden, so the SDK base rejects them. A backend exception is `BadInternalError` for that node and a deadline is `BadTimeout`.
|
which are not overridden, so the SDK base rejects them. A backend exception is `BadInternalError` for that node and a deadline is `BadTimeout`.
|
||||||
No `AuditHistoryUpdateEvent` is emitted.
|
On a `SqlServer` database still at schema v1 every item is `BadHistoryOperationUnsupported` ([#674](#modified-value-history-part-11)).
|
||||||
|
No `AuditHistoryUpdateEvent` is emitted (#674.5, not implemented).
|
||||||
|
|
||||||
|
### Primary-only (#674)
|
||||||
|
|
||||||
|
HistoryUpdate is served only by the node holding the redundancy **Primary** data-plane role — the same verdict
|
||||||
|
(`IDriverPrimaryRoleSource`, published by `DriverHostActor`) that gates native-alarm acknowledges (#597). On any other node every node of
|
||||||
|
the request that **passed authorization** is refused **`BadInvalidState`** (node-level code, with a message pointing at the Primary) and
|
||||||
|
nothing reaches the backend. Authorization runs first, so an unauthorized caller learns `BadUserAccessDenied`, not the node's role.
|
||||||
|
HistoryRead is **not** gated. Why: rewriting history is a single-writer operation — both backends capture the prior value for the
|
||||||
|
[modification trail](#modified-value-history-part-11) under a per-tag, **per-process** lock (QuestDB) or statement (SqlServer), so two pair
|
||||||
|
nodes updating one tag concurrently could record a wrong prior value. A single-node deployment (no redundancy manager) is unaffected. Clients
|
||||||
|
should send HistoryUpdate to the server advertising the higher `ServiceLevel`.
|
||||||
|
|
||||||
### The `HistoryWrite` AccessLevel bit
|
### The `HistoryWrite` AccessLevel bit
|
||||||
|
|
||||||
|
|||||||
@@ -32,13 +32,13 @@ Single-node dev: `OTOPCUA_ROLES=admin,driver` (or the `Cluster:Roles` equivalent
|
|||||||
|
|
||||||
### Per-role configuration overlays
|
### Per-role configuration overlays
|
||||||
|
|
||||||
`Program.cs` builds a role suffix by joining the resolved `EffectiveRoles.Gossiped` roles **alphabetically** with `-` and loads `appsettings.{roleSuffix}.json` as an optional overlay on top of base `appsettings.json`. A plain role (`admin`/`driver`/`dev`) is always lower-case in `Gossiped` regardless of how the operator spelled it, so the overlay filename for those is stable (`appsettings.driver.json`); if a `cluster-{ClusterId}` role is also configured, its suffix carries into the filename **verbatim** (e.g. `Cluster:Roles: ["driver", "cluster-SITE-A"]` loads `appsettings.cluster-SITE-A-driver.json`), since `Gossiped` preserves that spelling by design (see [Configuration.md § `Roles`](Configuration.md#appsettings-sections)). Three overlays ship in `src/Server/ZB.MOM.WW.OtOpcUa.Host/`:
|
`Program.cs` builds a role suffix (`RoleOverlay.Suffix`) by joining the resolved `EffectiveRoles.Gossiped` **plain** roles (`admin`/`driver`/`dev`) alphabetically with `-` and loads `appsettings.{roleSuffix}.json` as an optional overlay on top of base `appsettings.json`. A plain role is always lower-case in `Gossiped` regardless of how the operator spelled it, so the overlay filename is stable (`appsettings.driver.json`). **`cluster-{ClusterId}` roles are NOT part of the filename** (#678.6): they are mesh-membership tags, so `Cluster:Roles: ["driver", "cluster-SITE-A"]` loads `appsettings.driver.json` and `["admin", "driver", "cluster-MAIN"]` loads `appsettings.admin-driver.json`. (Before #678.6 the cluster role was folded into the name, producing e.g. `appsettings.admin-cluster-MAIN-driver.json`, which never exists — so on every node with a cluster role no shipped overlay ever loaded.) Three overlays ship in `src/Server/ZB.MOM.WW.OtOpcUa.Host/`:
|
||||||
|
|
||||||
- `appsettings.admin.json` — admin-only nodes
|
- `appsettings.admin.json` — admin-only nodes
|
||||||
- `appsettings.driver.json` — driver-only nodes
|
- `appsettings.driver.json` — driver-only nodes
|
||||||
- `appsettings.admin-driver.json` — fused single-node dev / small deployments
|
- `appsettings.admin-driver.json` — fused single-node dev / small deployments
|
||||||
|
|
||||||
All three carry Serilog log-level overrides + `Security:Ldap:DevStubMode = false`. Configuration loading order (lowest to highest precedence) is:
|
All three carry Serilog log-level overrides + `Security:Ldap:DevStubMode = false` + `Security:Ldap:Transport = Ldaps` (audit table: [Configuration.md § Role overlays](Configuration.md#role-overlay-audit)). Configuration loading order (lowest to highest precedence) is:
|
||||||
|
|
||||||
`appsettings.json` < `appsettings.{Environment}.json` < `appsettings.{role}.json` < environment variables < command-line args
|
`appsettings.json` < `appsettings.{Environment}.json` < `appsettings.{role}.json` < environment variables < command-line args
|
||||||
|
|
||||||
|
|||||||
@@ -884,6 +884,7 @@ HistoryUpdate rewrites recorded history, so unlike HistoryRead it is gated **wha
|
|||||||
- **Flag off (default):** the session's API key is re-checked on the call (`DataPlaneAuthorizer.AuthorizeKeyUsable`, Gitea #563 — an expired or disabled key loses its activation-time roles), then the session must carry the **`HistoryUpdate` role** (`OpcUaDataPlaneRoles.HistoryUpdate`). `ApiKeyInterimRoleMapper` confers it **only from a `Cluster`-scoped grant** carrying `NodePermissions.HistoryUpdate` — narrower-scoped grants confer nothing, for the same reason as `WriteOperate` (honouring a flag without honouring its scope would widen it). A key holding only `WriteOperate` is denied `BadUserAccessDenied`.
|
- **Flag off (default):** the session's API key is re-checked on the call (`DataPlaneAuthorizer.AuthorizeKeyUsable`, Gitea #563 — an expired or disabled key loses its activation-time roles), then the session must carry the **`HistoryUpdate` role** (`OpcUaDataPlaneRoles.HistoryUpdate`). `ApiKeyInterimRoleMapper` confers it **only from a `Cluster`-scoped grant** carrying `NodePermissions.HistoryUpdate` — narrower-scoped grants confer nothing, for the same reason as `WriteOperate` (honouring a flag without honouring its scope would widen it). A key holding only `WriteOperate` is denied `BadUserAccessDenied`.
|
||||||
- **Flag on:** `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)` decides per node and **replaces** the role check (never chained) — a session with no roles succeeds on the strength of a scoped `HistoryUpdate` grant, and a `WriteOperate` grant does not help.
|
- **Flag on:** `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)` decides per node and **replaces** the role check (never chained) — a session with no roles succeeds on the strength of a scoped `HistoryUpdate` grant, and a `WriteOperate` grant does not help.
|
||||||
- **Availability is separate from authorization.** A node accepts HistoryUpdate only if it is historized **and** the selected historian backend has an updater when the node materialises (the `HistoryWrite` AccessLevel bit). `Backend=Gateway` has none, so it answers `BadHistoryOperationUnsupported` for every node whatever the grant. Operations, result codes and unsupported cases: [`docs/Historian.md` §HistoryUpdate](Historian.md#historyupdate).
|
- **Availability is separate from authorization.** A node accepts HistoryUpdate only if it is historized **and** the selected historian backend has an updater when the node materialises (the `HistoryWrite` AccessLevel bit). `Backend=Gateway` has none, so it answers `BadHistoryOperationUnsupported` for every node whatever the grant. Operations, result codes and unsupported cases: [`docs/Historian.md` §HistoryUpdate](Historian.md#historyupdate).
|
||||||
|
- **Primary-only, after authorization (Gitea #674).** An authorized HistoryUpdate on a node that is not the redundancy Primary is refused `BadInvalidState` (the native-alarm-ack verdict, #597); an unauthorized one is told `BadUserAccessDenied` first, so the refusal never reveals the node's role to a caller who may not update.
|
||||||
- **Every applied change is attributed (Part 11 modified-value history, Gitea #668).** The session's principal (the API key's
|
- **Every applied change is attributed (Part 11 modified-value history, Gitea #668).** The session's principal (the API key's
|
||||||
`NodeAcl` principal, or the reserved `anonymous`) and key id are passed to the backend with every HistoryUpdate and recorded in the
|
`NodeAcl` principal, or the reserved `anonymous`) and key id are passed to the backend with every HistoryUpdate and recorded in the
|
||||||
modification trail next to the prior value; a modified-value HistoryRead (`IsReadModified = true`) returns it as
|
modification trail next to the prior value; a modified-value HistoryRead (`IsReadModified = true`) returns it as
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ public class HistoryReadCommand : CommandBase
|
|||||||
/// <summary>
|
/// <summary>
|
||||||
/// Gets the optional aggregate name used when the operator wants processed history instead of raw samples.
|
/// Gets the optional aggregate name used when the operator wants processed history instead of raw samples.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
[CommandOption("aggregate", Description = "Aggregate function: Average, Minimum, Maximum, Count, Start, End, StandardDeviation")]
|
[CommandOption("aggregate", Description = "Aggregate function: Average, Minimum, Maximum, Count, Start, End, StandardDeviation, TimeAverage, TimeAverage2, Total, Total2, Interpolative")]
|
||||||
public string? Aggregate { get; init; }
|
public string? Aggregate { get; init; }
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -183,6 +183,12 @@ public class HistoryReadCommand : CommandBase
|
|||||||
await console.Output.WriteLineAsync();
|
await console.Output.WriteLineAsync();
|
||||||
await console.Output.WriteLineAsync($"{values.Count} values returned.");
|
await console.Output.WriteLineAsync($"{values.Count} values returned.");
|
||||||
}
|
}
|
||||||
|
catch (ServiceResultException ex) when (ex.InnerException is null)
|
||||||
|
{
|
||||||
|
// e.g. HistoryReadStatus.ThrowIfRefused: a refusal is an operator-facing outcome, not a crash.
|
||||||
|
// A wrapped failure (certificate validation, identity rejection on connect) keeps its full detail.
|
||||||
|
throw new CommandException(ex.Message);
|
||||||
|
}
|
||||||
finally
|
finally
|
||||||
{
|
{
|
||||||
if (service != null)
|
if (service != null)
|
||||||
@@ -204,8 +210,13 @@ public class HistoryReadCommand : CommandBase
|
|||||||
"start" or "first" => AggregateType.Start,
|
"start" or "first" => AggregateType.Start,
|
||||||
"end" or "last" => AggregateType.End,
|
"end" or "last" => AggregateType.End,
|
||||||
"standarddeviation" or "stddev" or "stdev" => AggregateType.StandardDeviation,
|
"standarddeviation" or "stddev" or "stdev" => AggregateType.StandardDeviation,
|
||||||
|
"timeaverage" or "timeavg" => AggregateType.TimeAverage,
|
||||||
|
"timeaverage2" or "timeavg2" => AggregateType.TimeAverage2,
|
||||||
|
"total" => AggregateType.Total,
|
||||||
|
"total2" => AggregateType.Total2,
|
||||||
|
"interpolative" or "interp" => AggregateType.Interpolative,
|
||||||
_ => throw new ArgumentException(
|
_ => throw new ArgumentException(
|
||||||
$"Unknown aggregate: '{name}'. Supported: Average, Minimum, Maximum, Count, Start, End, StandardDeviation")
|
$"Unknown aggregate: '{name}'. Supported: Average, Minimum, Maximum, Count, Start, End, StandardDeviation, TimeAverage, TimeAverage2, Total, Total2, Interpolative")
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -85,17 +85,22 @@ internal static class EndpointSelector
|
|||||||
$"No endpoint found with security mode '{requestedMode}'. Available endpoints: {available}");
|
$"No endpoint found with security mode '{requestedMode}'. Available endpoints: {available}");
|
||||||
}
|
}
|
||||||
|
|
||||||
// Rewrite endpoint URL hostname to match user-supplied hostname. Necessary
|
// Rewrite the endpoint URL host AND port to the user-supplied ones (as the SDK's
|
||||||
// when the OPC UA server returns a discovery URL using a different hostname
|
// CoreClientUtils.SelectEndpoint does). A server behind a port mapping or internal DNS
|
||||||
// (e.g. internal DNS name) than the one the operator routed to.
|
// advertises its own host:port (docker-dev containers all say :4840); following that would
|
||||||
|
// open the session on a different server and fail the secure-channel certificate check.
|
||||||
|
// Only when the scheme matches: a different scheme (https vs opc.tcp) is a different transport.
|
||||||
var serverUri = new Uri(best.EndpointUrl);
|
var serverUri = new Uri(best.EndpointUrl);
|
||||||
var requestedUri = new Uri(endpointUrl);
|
var requestedUri = new Uri(endpointUrl);
|
||||||
if (serverUri.Host != requestedUri.Host)
|
// A URL typed without a port (opc.tcp has no registered default, so Port is -1) keeps the advertised port.
|
||||||
|
var port = requestedUri.Port > 0 ? requestedUri.Port : serverUri.Port;
|
||||||
|
if (string.Equals(serverUri.Scheme, requestedUri.Scheme, StringComparison.OrdinalIgnoreCase)
|
||||||
|
&& (serverUri.Host != requestedUri.Host || serverUri.Port != port))
|
||||||
{
|
{
|
||||||
var builder = new UriBuilder(best.EndpointUrl) { Host = requestedUri.Host };
|
var builder = new UriBuilder(best.EndpointUrl) { Host = requestedUri.Host, Port = port };
|
||||||
best.EndpointUrl = builder.ToString();
|
best.EndpointUrl = builder.ToString();
|
||||||
Logger.Debug("Rewrote endpoint host from {ServerHost} to {RequestedHost}", serverUri.Host,
|
Logger.Debug("Rewrote endpoint from {ServerHost}:{ServerPort} to {RequestedHost}:{RequestedPort}",
|
||||||
requestedUri.Host);
|
serverUri.Host, serverUri.Port, requestedUri.Host, port);
|
||||||
}
|
}
|
||||||
|
|
||||||
return best;
|
return best;
|
||||||
|
|||||||
@@ -24,6 +24,11 @@ public static class AggregateTypeMapper
|
|||||||
AggregateType.Start => ObjectIds.AggregateFunction_Start,
|
AggregateType.Start => ObjectIds.AggregateFunction_Start,
|
||||||
AggregateType.End => ObjectIds.AggregateFunction_End,
|
AggregateType.End => ObjectIds.AggregateFunction_End,
|
||||||
AggregateType.StandardDeviation => ObjectIds.AggregateFunction_StandardDeviationPopulation,
|
AggregateType.StandardDeviation => ObjectIds.AggregateFunction_StandardDeviationPopulation,
|
||||||
|
AggregateType.TimeAverage => ObjectIds.AggregateFunction_TimeAverage,
|
||||||
|
AggregateType.TimeAverage2 => ObjectIds.AggregateFunction_TimeAverage2,
|
||||||
|
AggregateType.Total => ObjectIds.AggregateFunction_Total,
|
||||||
|
AggregateType.Total2 => ObjectIds.AggregateFunction_Total2,
|
||||||
|
AggregateType.Interpolative => ObjectIds.AggregateFunction_Interpolative,
|
||||||
_ => throw new ArgumentOutOfRangeException(nameof(aggregate), aggregate, "Unknown AggregateType value.")
|
_ => throw new ArgumentOutOfRangeException(nameof(aggregate), aggregate, "Unknown AggregateType value.")
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -24,5 +24,20 @@ public enum AggregateType
|
|||||||
End,
|
End,
|
||||||
|
|
||||||
/// <summary>Population standard deviation of values in the interval.</summary>
|
/// <summary>Population standard deviation of values in the interval.</summary>
|
||||||
StandardDeviation
|
StandardDeviation,
|
||||||
|
|
||||||
|
/// <summary>Time-weighted average, interpolated bounds (Part 13 TimeAverage).</summary>
|
||||||
|
TimeAverage,
|
||||||
|
|
||||||
|
/// <summary>Time-weighted average, stepped bounds (Part 13 TimeAverage2).</summary>
|
||||||
|
TimeAverage2,
|
||||||
|
|
||||||
|
/// <summary>Time-weighted integral over the interval (Part 13 Total).</summary>
|
||||||
|
Total,
|
||||||
|
|
||||||
|
/// <summary>Time-weighted integral, stepped bounds (Part 13 Total2).</summary>
|
||||||
|
Total2,
|
||||||
|
|
||||||
|
/// <summary>Interpolated value at the interval start (Part 13 Interpolative).</summary>
|
||||||
|
Interpolative
|
||||||
}
|
}
|
||||||
@@ -53,7 +53,12 @@ public partial class HistoryViewModel : ObservableObject
|
|||||||
AggregateType.Count,
|
AggregateType.Count,
|
||||||
AggregateType.Start,
|
AggregateType.Start,
|
||||||
AggregateType.End,
|
AggregateType.End,
|
||||||
AggregateType.StandardDeviation
|
AggregateType.StandardDeviation,
|
||||||
|
AggregateType.TimeAverage,
|
||||||
|
AggregateType.TimeAverage2,
|
||||||
|
AggregateType.Total,
|
||||||
|
AggregateType.Total2,
|
||||||
|
AggregateType.Interpolative
|
||||||
];
|
];
|
||||||
|
|
||||||
/// <summary>Gets a value indicating whether an aggregate read is selected.</summary>
|
/// <summary>Gets a value indicating whether an aggregate read is selected.</summary>
|
||||||
|
|||||||
@@ -88,7 +88,7 @@ public sealed class HistoryReadLimitExceededException : InvalidOperationExceptio
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// One row returned by <see cref="IHistorianDataSource.ReadEventsAsync"/> — a historical
|
/// One row returned by <see cref="IHistorianDataSource.ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/> — a historical
|
||||||
/// alarm/event record, not the OPC UA live-event stream. Fields match the minimum set the
|
/// alarm/event record, not the OPC UA live-event stream. Fields match the minimum set the
|
||||||
/// Server needs to populate a <c>HistoryEventFieldList</c> for HistoryReadEvents responses.
|
/// Server needs to populate a <c>HistoryEventFieldList</c> for HistoryReadEvents responses.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
@@ -106,8 +106,8 @@ public sealed record HistoricalEvent(
|
|||||||
string? Message,
|
string? Message,
|
||||||
ushort Severity);
|
ushort Severity);
|
||||||
|
|
||||||
/// <summary>Result of a <see cref="IHistorianDataSource.ReadEventsAsync"/> call.</summary>
|
/// <summary>Result of a <see cref="IHistorianDataSource.ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/> call.</summary>
|
||||||
/// <param name="Events">Events in chronological order by <c>EventTimeUtc</c>.</param>
|
/// <param name="Events">Events ordered by <c>EventTimeUtc</c> in the read's direction — ascending, or descending for a reverse read (#675).</param>
|
||||||
/// <param name="ContinuationPoint">Opaque token for the next call when more events are available; null when complete.</param>
|
/// <param name="ContinuationPoint">Opaque token for the next call when more events are available; null when complete.</param>
|
||||||
public sealed record HistoricalEventsResult(
|
public sealed record HistoricalEventsResult(
|
||||||
IReadOnlyList<HistoricalEvent> Events,
|
IReadOnlyList<HistoricalEvent> Events,
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ public interface IHistorianDataSource : IDisposable
|
|||||||
/// <c>uint maxValuesPerNode</c> because their OPC UA HistoryRead surface has no
|
/// <c>uint maxValuesPerNode</c> because their OPC UA HistoryRead surface has no
|
||||||
/// equivalent "use default" sentinel.
|
/// equivalent "use default" sentinel.
|
||||||
///
|
///
|
||||||
/// This surface declares <see cref="ReadAtTimeAsync"/> and <see cref="ReadEventsAsync"/>
|
/// This surface declares <see cref="ReadAtTimeAsync"/> and <see cref="ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/>
|
||||||
/// as required members — a server-side historian owns the full read surface.
|
/// as required members — a server-side historian owns the full read surface.
|
||||||
/// </remarks>
|
/// </remarks>
|
||||||
/// <param name="sourceName">The source name to filter events, or null to return events from all sources.</param>
|
/// <param name="sourceName">The source name to filter events, or null to return events from all sources.</param>
|
||||||
@@ -101,6 +101,46 @@ public interface IHistorianDataSource : IDisposable
|
|||||||
int maxEvents,
|
int maxEvents,
|
||||||
CancellationToken cancellationToken);
|
CancellationToken cancellationToken);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Read historical alarm / event records in a chosen direction — the overload the server's HistoryReadEvents
|
||||||
|
/// paging calls (Gitea #675).
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// The bounds may be given in either order; let <c>lo</c>/<c>hi</c> be the earlier/later of them. Forward
|
||||||
|
/// (<paramref name="reverse"/> <c>false</c>) is exactly <see cref="ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/>:
|
||||||
|
/// <c>[lo, hi)</c> ascending by <c>(EventTimeUtc, EventId)</c>. Reverse is the Part 11 reverse window
|
||||||
|
/// <c>(lo, hi]</c> in DESCENDING <c>(EventTimeUtc, EventId)</c> order, so a cap keeps the LATEST
|
||||||
|
/// <paramref name="maxEvents"/> events — the mirror of forward, with the same tie-breaker reversed, so a
|
||||||
|
/// resumed reverse page that re-reads its boundary timestamp sees the boundary ties in the same order.
|
||||||
|
/// <paramref name="maxEvents"/> and the continuation-point contract are as for the forward read.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// The default forwards a forward read to the five-argument overload and throws
|
||||||
|
/// <see cref="NotSupportedException"/> for a reverse one — a backend that cannot read newest-first (Gateway,
|
||||||
|
/// Null) — which the server reports as <c>BadHistoryOperationUnsupported</c> rather than returning the
|
||||||
|
/// earliest events in ascending order as if they were the answer.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
/// <param name="sourceName">The source name to filter events, or null to return events from all sources.</param>
|
||||||
|
/// <param name="startUtc">One bound of the window (UTC).</param>
|
||||||
|
/// <param name="endUtc">The other bound of the window (UTC).</param>
|
||||||
|
/// <param name="maxEvents">The cap; non-positive uses the backend default (see the five-argument overload).</param>
|
||||||
|
/// <param name="reverse"><c>true</c> to read newest-first over <c>(lo, hi]</c>.</param>
|
||||||
|
/// <param name="cancellationToken">Cancellation.</param>
|
||||||
|
/// <returns>The matching events in the read's direction, with a continuation point if truncated.</returns>
|
||||||
|
/// <exception cref="NotSupportedException">A reverse read on a backend that cannot read newest-first.</exception>
|
||||||
|
Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName,
|
||||||
|
DateTime startUtc,
|
||||||
|
DateTime endUtc,
|
||||||
|
int maxEvents,
|
||||||
|
bool reverse,
|
||||||
|
CancellationToken cancellationToken) =>
|
||||||
|
reverse
|
||||||
|
? throw new NotSupportedException("This historian backend cannot read events newest-first (reverse).")
|
||||||
|
: ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken);
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Point-in-time health snapshot for diagnostics and dashboards. Pure
|
/// Point-in-time health snapshot for diagnostics and dashboards. Pure
|
||||||
/// observation; never blocks on backend I/O.
|
/// observation; never blocks on backend I/O.
|
||||||
|
|||||||
@@ -125,6 +125,21 @@ public sealed class GatewayHistorianDataSource : IHistorianDataSource, IAsyncDis
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
/// <remarks>
|
||||||
|
/// A reverse (newest-first) read is refused with <see cref="NotSupportedException"/> — the server reports
|
||||||
|
/// <c>BadHistoryOperationUnsupported</c> (Gitea #675). The gateway's <c>ReadEvents</c> streams ascending only and
|
||||||
|
/// rejects <c>end < start</c>, so the only way to serve "the latest N" would be to drain the whole window and
|
||||||
|
/// keep its tail, an unbounded read the cap exists to prevent. Nothing is sent.
|
||||||
|
/// </remarks>
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse,
|
||||||
|
CancellationToken cancellationToken) =>
|
||||||
|
reverse
|
||||||
|
? throw new NotSupportedException(
|
||||||
|
"The HistorianGateway backend reads events oldest-first only; a reverse (newest-first) events read is not supported.")
|
||||||
|
: ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken);
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||||
|
|||||||
+13
-3
@@ -361,8 +361,18 @@ public sealed class QuestDbHistorianDataSource : IHistorianDataSource
|
|||||||
/// <c>maxEvents + 1</c> rows are fetched; when the extra row arrives it is dropped and the continuation point is
|
/// <c>maxEvents + 1</c> rows are fetched; when the extra row arrives it is dropped and the continuation point is
|
||||||
/// the UTF-8 of <see cref="EventContinuation.Encode"/> for that first not-returned event.
|
/// the UTF-8 of <see cref="EventContinuation.Encode"/> for that first not-returned event.
|
||||||
/// </remarks>
|
/// </remarks>
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken) =>
|
||||||
|
ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, reverse: false, cancellationToken);
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
/// <remarks>
|
||||||
|
/// Forward is the five-argument read. Reverse (Gitea #675) is the (rounded) <c>(lo, hi]</c> ordered by
|
||||||
|
/// <c>(ts, event_id)</c> DESCENDING, so <c>LIMIT</c> keeps the latest events; truncation and the continuation point
|
||||||
|
/// work as forward.
|
||||||
|
/// </remarks>
|
||||||
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken)
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse, CancellationToken cancellationToken)
|
||||||
{
|
{
|
||||||
// No stored event can carry a source longer than the limit, so an over-long filter matches nothing.
|
// No stored event can carry a source longer than the limit, so an over-long filter matches nothing.
|
||||||
if (sourceName is not null && sourceName.Length > HistorianTagNames.MaxLength)
|
if (sourceName is not null && sourceName.Length > HistorianTagNames.MaxLength)
|
||||||
@@ -376,9 +386,9 @@ public sealed class QuestDbHistorianDataSource : IHistorianDataSource
|
|||||||
var bounded = maxEvents > 0 && maxEvents < int.MaxValue;
|
var bounded = maxEvents > 0 && maxEvents < int.MaxValue;
|
||||||
var sql =
|
var sql =
|
||||||
$"SELECT event_id, source, ts, received_utc, message, severity FROM {_conn.EventTable} " +
|
$"SELECT event_id, source, ts, received_utc, message, severity FROM {_conn.EventTable} " +
|
||||||
"WHERE ts >= $1 AND ts < $2" +
|
(reverse ? "WHERE ts > $1 AND ts <= $2" : "WHERE ts >= $1 AND ts < $2") +
|
||||||
(sourceName is null ? "" : " AND source = $3") +
|
(sourceName is null ? "" : " AND source = $3") +
|
||||||
" ORDER BY ts ASC, event_id ASC" +
|
(reverse ? " ORDER BY ts DESC, event_id DESC" : " ORDER BY ts ASC, event_id ASC") +
|
||||||
(bounded ? (sourceName is null ? " LIMIT $3" : " LIMIT $4") : "");
|
(bounded ? (sourceName is null ? " LIMIT $3" : " LIMIT $4") : "");
|
||||||
|
|
||||||
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
||||||
|
|||||||
@@ -137,7 +137,10 @@ internal sealed class QuestDbSchemaBootstrapper
|
|||||||
// The ALTER is a WAL transaction: tables() keeps the old value until it applies, so do NOT re-read to verify.
|
// The ALTER is a WAL transaction: tables() keeps the old value until it applies, so do NOT re-read to verify.
|
||||||
await using (var alter = _conn.CreateCommand(conn, QuestDbTables.AlterTtl(table, _options.RetentionDays)))
|
await using (var alter = _conn.CreateCommand(conn, QuestDbTables.AlterTtl(table, _options.RetentionDays)))
|
||||||
await alter.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
|
await alter.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
|
||||||
_log.LogInformation("Applied TTL {Days} day(s) to QuestDB table {Table}.", _options.RetentionDays, table);
|
if (_options.RetentionDays is { } days)
|
||||||
|
_log.LogInformation("Applied TTL {Days} day(s) to QuestDB table {Table}.", days, table);
|
||||||
|
else
|
||||||
|
_log.LogInformation("Cleared TTL on QuestDB table {Table}.", table);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -27,4 +27,10 @@ public static class HistorianStatusCodes
|
|||||||
/// status, or a tag name longer than <see cref="HistorianTagNames.MaxLength"/>.
|
/// status, or a tag name longer than <see cref="HistorianTagNames.MaxLength"/>.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public const uint BadInvalidArgument = 0x80AB0000u;
|
public const uint BadInvalidArgument = 0x80AB0000u;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A modified-value read or HistoryUpdate item against a SQL Server schema still at version 1 (no Part 11
|
||||||
|
/// modification trail, Gitea #674): refused without touching the database.
|
||||||
|
/// </summary>
|
||||||
|
public const uint BadHistoryOperationUnsupported = 0x80720000u;
|
||||||
}
|
}
|
||||||
|
|||||||
+21
-4
@@ -154,6 +154,12 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
|||||||
var bounded = maxValues > 0 && maxValues < int.MaxValue;
|
var bounded = maxValues > 0 && maxValues < int.MaxValue;
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
|
// #674: a v1 schema has no SampleModified. NotSupportedException is the interface's "no trail" signal (the
|
||||||
|
// node manager reports BadHistoryOperationUnsupported); it is not a backend failure and sends no query.
|
||||||
|
await _bootstrap.EnsureAsync(cancellationToken).ConfigureAwait(false);
|
||||||
|
if (!_bootstrap.SupportsModifiedTrail)
|
||||||
|
throw new NotSupportedException(
|
||||||
|
$"Historian schema [{_conn.Schema}] is below version {SqlServerSchemaBootstrapper.ModifiedTrailVersion}: no modified-value history.");
|
||||||
var values = new List<HistoryModifiedValue>();
|
var values = new List<HistoryModifiedValue>();
|
||||||
var hasMore = false;
|
var hasMore = false;
|
||||||
var tagId = await ResolveAsync(fullReference, cancellationToken).ConfigureAwait(false);
|
var tagId = await ResolveAsync(fullReference, cancellationToken).ConfigureAwait(false);
|
||||||
@@ -219,7 +225,7 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
|||||||
RecordSuccess();
|
RecordSuccess();
|
||||||
return new HistoryModifiedReadResult(values, hasMore);
|
return new HistoryModifiedReadResult(values, hasMore);
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex) when (ex is not NotSupportedException)
|
||||||
{
|
{
|
||||||
RecordFailure(ex);
|
RecordFailure(ex);
|
||||||
throw;
|
throw;
|
||||||
@@ -314,8 +320,18 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
|||||||
/// of <c>"{EventTimeUtc.Ticks}|{EventId}"</c> of that first not-returned event (opaque to callers; only
|
/// of <c>"{EventTimeUtc.Ticks}|{EventId}"</c> of that first not-returned event (opaque to callers; only
|
||||||
/// its presence is contractual).
|
/// its presence is contractual).
|
||||||
/// </remarks>
|
/// </remarks>
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken) =>
|
||||||
|
ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, reverse: false, cancellationToken);
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
/// <remarks>
|
||||||
|
/// Forward is the five-argument read. Reverse (Gitea #675) is <c>(lo, hi]</c> ordered by
|
||||||
|
/// <c>(EventTimeUtc, EventId)</c> DESCENDING — the clustered index scanned backwards — so <c>TOP</c> keeps the latest
|
||||||
|
/// events; truncation and the continuation point work as forward.
|
||||||
|
/// </remarks>
|
||||||
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, CancellationToken cancellationToken)
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse, CancellationToken cancellationToken)
|
||||||
{
|
{
|
||||||
// No stored event can carry a source longer than the column (the alarm writer dead-letters it rather than
|
// No stored event can carry a source longer than the column (the alarm writer dead-letters it rather than
|
||||||
// truncating), so an over-long filter matches nothing — never a truncated prefix of someone else's source.
|
// truncating), so an over-long filter matches nothing — never a truncated prefix of someone else's source.
|
||||||
@@ -328,9 +344,10 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
|||||||
var bounded = maxEvents > 0 && maxEvents < int.MaxValue;
|
var bounded = maxEvents > 0 && maxEvents < int.MaxValue;
|
||||||
var sql =
|
var sql =
|
||||||
$"SELECT {(bounded ? "TOP (@take) " : "")}EventId, SourceName, EventTimeUtc, ReceivedTimeUtc, Message, Severity " +
|
$"SELECT {(bounded ? "TOP (@take) " : "")}EventId, SourceName, EventTimeUtc, ReceivedTimeUtc, Message, Severity " +
|
||||||
$"FROM {_conn.Q("Event")} WHERE EventTimeUtc >= @lo AND EventTimeUtc < @hi" +
|
$"FROM {_conn.Q("Event")} WHERE " +
|
||||||
|
(reverse ? "EventTimeUtc > @lo AND EventTimeUtc <= @hi" : "EventTimeUtc >= @lo AND EventTimeUtc < @hi") +
|
||||||
(sourceName is null ? "" : " AND SourceName = @source") +
|
(sourceName is null ? "" : " AND SourceName = @source") +
|
||||||
" ORDER BY EventTimeUtc ASC, EventId ASC";
|
(reverse ? " ORDER BY EventTimeUtc DESC, EventId DESC" : " ORDER BY EventTimeUtc ASC, EventId ASC");
|
||||||
|
|
||||||
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
||||||
await using var cmd = _conn.CreateCommand(conn, sql);
|
await using var cmd = _conn.CreateCommand(conn, sql);
|
||||||
|
|||||||
+27
-1
@@ -27,21 +27,44 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
|||||||
/// value from <c>deleted.*</c> (NULL for an insert), the caller's <see cref="HistoryUpdateContext"/>, the
|
/// value from <c>deleted.*</c> (NULL for an insert), the caller's <see cref="HistoryUpdateContext"/>, the
|
||||||
/// <see cref="HistoryModificationType"/> and <c>SYSUTCDATETIME()</c>. One statement means the sample change and its
|
/// <see cref="HistoryModificationType"/> and <c>SYSUTCDATETIME()</c>. One statement means the sample change and its
|
||||||
/// audit row commit or roll back together: a failed update leaves no audit row.</para>
|
/// audit row commit or roll back together: a failed update leaves no audit row.</para>
|
||||||
|
/// <para><b>Schema v1 (Gitea #674).</b> Without the trail (<see cref="SqlServerSchemaBootstrapper.SupportsModifiedTrail"/>
|
||||||
|
/// false) a change could not be audited, so every item (and DeleteRaw's single result) reports
|
||||||
|
/// <c>BadHistoryOperationUnsupported</c> and nothing is sent to the database beyond the one-time schema check.</para>
|
||||||
/// </summary>
|
/// </summary>
|
||||||
internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||||
{
|
{
|
||||||
private readonly SqlHistorianConnection _conn;
|
private readonly SqlHistorianConnection _conn;
|
||||||
private readonly SqlServerTagCatalog _tags;
|
private readonly SqlServerTagCatalog _tags;
|
||||||
|
private readonly SqlServerSchemaBootstrapper _bootstrap;
|
||||||
|
|
||||||
/// <summary>Creates the updater.</summary>
|
/// <summary>Creates the updater.</summary>
|
||||||
/// <param name="conn">Connection helper.</param>
|
/// <param name="conn">Connection helper.</param>
|
||||||
|
/// <param name="bootstrap">Schema bootstrapper — decides whether the modification trail exists (#674).</param>
|
||||||
/// <param name="tags">Tag catalog (runs the schema bootstrap itself).</param>
|
/// <param name="tags">Tag catalog (runs the schema bootstrap itself).</param>
|
||||||
public SqlServerHistorianDataUpdater(SqlHistorianConnection conn, SqlServerTagCatalog tags)
|
public SqlServerHistorianDataUpdater(SqlHistorianConnection conn, SqlServerSchemaBootstrapper bootstrap,
|
||||||
|
SqlServerTagCatalog tags)
|
||||||
{
|
{
|
||||||
_conn = conn;
|
_conn = conn;
|
||||||
|
_bootstrap = bootstrap;
|
||||||
_tags = tags;
|
_tags = tags;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>#674: ensures the schema and reports whether the Part 11 trail exists (no SQL after the first success).</summary>
|
||||||
|
/// <param name="ct">Cancellation.</param>
|
||||||
|
/// <returns>True when the modification trail is available.</returns>
|
||||||
|
private async Task<bool> TrailAvailableAsync(CancellationToken ct)
|
||||||
|
{
|
||||||
|
await _bootstrap.EnsureAsync(ct).ConfigureAwait(false);
|
||||||
|
return _bootstrap.SupportsModifiedTrail;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static uint[] Unsupported(int count)
|
||||||
|
{
|
||||||
|
var codes = new uint[count];
|
||||||
|
Array.Fill(codes, HistorianStatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
return codes;
|
||||||
|
}
|
||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public async Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
|
public async Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
|
||||||
IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context, CancellationToken ct)
|
IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context, CancellationToken ct)
|
||||||
@@ -49,6 +72,7 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
|||||||
ArgumentNullException.ThrowIfNull(tag);
|
ArgumentNullException.ThrowIfNull(tag);
|
||||||
ArgumentNullException.ThrowIfNull(values);
|
ArgumentNullException.ThrowIfNull(values);
|
||||||
ArgumentNullException.ThrowIfNull(context);
|
ArgumentNullException.ThrowIfNull(context);
|
||||||
|
if (!await TrailAvailableAsync(ct).ConfigureAwait(false)) return Unsupported(values.Count);
|
||||||
var plan = HistoryUpdatePlanner.Plan(tag, mode, values);
|
var plan = HistoryUpdatePlanner.Plan(tag, mode, values);
|
||||||
if (plan.Winners.Count == 0) return HistoryUpdatePlanner.Complete(plan, mode, new Dictionary<long, UpsertOutcome>());
|
if (plan.Winners.Count == 0) return HistoryUpdatePlanner.Complete(plan, mode, new Dictionary<long, UpsertOutcome>());
|
||||||
|
|
||||||
@@ -68,6 +92,7 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
|||||||
{
|
{
|
||||||
ArgumentNullException.ThrowIfNull(tag);
|
ArgumentNullException.ThrowIfNull(tag);
|
||||||
ArgumentNullException.ThrowIfNull(context);
|
ArgumentNullException.ThrowIfNull(context);
|
||||||
|
if (!await TrailAvailableAsync(ct).ConfigureAwait(false)) return HistorianStatusCodes.BadHistoryOperationUnsupported;
|
||||||
if (!HistorianTagNames.IsValid(tag)) return HistorianStatusCodes.BadInvalidArgument;
|
if (!HistorianTagNames.IsValid(tag)) return HistorianStatusCodes.BadInvalidArgument;
|
||||||
var tagId = await _tags.TryGetTagIdAsync(tag, ct).ConfigureAwait(false);
|
var tagId = await _tags.TryGetTagIdAsync(tag, ct).ConfigureAwait(false);
|
||||||
if (tagId is null) return HistoryUpdatePlanner.DeleteRawResult(tagKnown: false, 0);
|
if (tagId is null) return HistoryUpdatePlanner.DeleteRawResult(tagKnown: false, 0);
|
||||||
@@ -95,6 +120,7 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
|||||||
ArgumentNullException.ThrowIfNull(tag);
|
ArgumentNullException.ThrowIfNull(tag);
|
||||||
ArgumentNullException.ThrowIfNull(timesUtc);
|
ArgumentNullException.ThrowIfNull(timesUtc);
|
||||||
ArgumentNullException.ThrowIfNull(context);
|
ArgumentNullException.ThrowIfNull(context);
|
||||||
|
if (!await TrailAvailableAsync(ct).ConfigureAwait(false)) return Unsupported(timesUtc.Count);
|
||||||
if (!HistorianTagNames.IsValid(tag))
|
if (!HistorianTagNames.IsValid(tag))
|
||||||
{
|
{
|
||||||
var refused = new uint[timesUtc.Count];
|
var refused = new uint[timesUtc.Count];
|
||||||
|
|||||||
+4
-2
@@ -18,8 +18,10 @@ public sealed partial class SqlServerHistorianOptions
|
|||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// <c>true</c> creates the schema and applies missing scripts on first use; <c>false</c> only verifies
|
/// <c>true</c> creates the schema and applies missing scripts on first use; <c>false</c> only verifies
|
||||||
/// that the required schema version (<c>SqlServerSchemaBootstrapper.RequiredVersion</c>, currently 2) exists and
|
/// that the base schema version (<c>SqlServerSchemaBootstrapper.RequiredVersion</c>, 1) exists and fails every
|
||||||
/// fails every historian call with a clear error otherwise.
|
/// historian call with a clear error otherwise. A verified v1 schema (no Part 11 modification trail, script 002
|
||||||
|
/// not applied) is served with modified reads and HistoryUpdate refused <c>BadHistoryOperationUnsupported</c>
|
||||||
|
/// and one Warning per boot (Gitea #674).
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public bool AutoCreateSchema { get; set; } = true;
|
public bool AutoCreateSchema { get; set; } = true;
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -99,7 +99,7 @@ public sealed class SqlServerHistorianProvider : IHistorianBackendProvider
|
|||||||
public IHistorianDataUpdater? CreateUpdater(IServiceProvider sp)
|
public IHistorianDataUpdater? CreateUpdater(IServiceProvider sp)
|
||||||
{
|
{
|
||||||
var s = Get(sp);
|
var s = Get(sp);
|
||||||
return new SqlServerHistorianDataUpdater(s.Conn, s.Catalog);
|
return new SqlServerHistorianDataUpdater(s.Conn, s.Bootstrap, s.Catalog);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
|
|||||||
+26
-12
@@ -11,7 +11,8 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
|||||||
/// <c>DELETE TOP(n)</c> batches (under the lock-escalation threshold). Samples are purged per tag so each
|
/// <c>DELETE TOP(n)</c> batches (under the lock-escalation threshold). Samples are purged per tag so each
|
||||||
/// delete is a seek on the clustered key <c>(TagId, SourceTimeUtc)</c>; events seek on <c>CX_Event_Time</c>.
|
/// delete is a seek on the clustered key <c>(TagId, SourceTimeUtc)</c>; events seek on <c>CX_Event_Time</c>.
|
||||||
/// A session <c>sp_getapplock</c> (timeout 0) serialises purgers that share one database; a loser skips.
|
/// A session <c>sp_getapplock</c> (timeout 0) serialises purgers that share one database; a loser skips.
|
||||||
/// To purge another table later, add it to <see cref="PurgeAsync"/>.
|
/// To purge another table later, add it to <see cref="PurgeAsync"/>. On a v1 schema (no Part 11 trail, Gitea #674)
|
||||||
|
/// the <c>SampleModified</c> purge is skipped and absent from the results.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
||||||
{
|
{
|
||||||
@@ -42,6 +43,9 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
|||||||
/// <summary>Test seam for <see cref="SqlConnection.ClearPool"/>.</summary>
|
/// <summary>Test seam for <see cref="SqlConnection.ClearPool"/>.</summary>
|
||||||
internal Action<SqlConnection> ClearPool { get; set; } = SqlConnection.ClearPool;
|
internal Action<SqlConnection> ClearPool { get; set; } = SqlConnection.ClearPool;
|
||||||
|
|
||||||
|
/// <summary>Test seam: the release statement. Tests point it at a failing statement to exercise the pool-clear fallback.</summary>
|
||||||
|
internal string ReleaseSql { get; set; } = "EXEC sp_releaseapplock @Resource = @res, @LockOwner = 'Session';";
|
||||||
|
|
||||||
public async Task<IReadOnlyList<RetentionPurgeResult>> PurgeAsync(CancellationToken ct)
|
public async Task<IReadOnlyList<RetentionPurgeResult>> PurgeAsync(CancellationToken ct)
|
||||||
{
|
{
|
||||||
await _bootstrap.EnsureAsync(ct).ConfigureAwait(false);
|
await _bootstrap.EnsureAsync(ct).ConfigureAwait(false);
|
||||||
@@ -69,6 +73,8 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
|||||||
return [];
|
return [];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// #674: a v1 schema has no SampleModified table — skip its purge rather than fail the whole pass.
|
||||||
|
var trail = _bootstrap.SupportsModifiedTrail;
|
||||||
long samples = 0, modified = 0;
|
long samples = 0, modified = 0;
|
||||||
var tagIds = new List<int>();
|
var tagIds = new List<int>();
|
||||||
await using (var cmd = _conn.CreateCommand(conn, $"SELECT TagId FROM {_conn.Q("Tag")}"))
|
await using (var cmd = _conn.CreateCommand(conn, $"SELECT TagId FROM {_conn.Q("Tag")}"))
|
||||||
@@ -82,21 +88,30 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
|||||||
cutoff, id, ct).ConfigureAwait(false);
|
cutoff, id, ct).ConfigureAwait(false);
|
||||||
// The Part 11 modification trail (#668) ages out on the SAME source-time cutoff as the samples it
|
// The Part 11 modification trail (#668) ages out on the SAME source-time cutoff as the samples it
|
||||||
// describes; a purge never writes to it (no trigger, no OUTPUT INTO).
|
// describes; a purge never writes to it (no trigger, no OUTPUT INTO).
|
||||||
modified += await DeleteLoopAsync(conn,
|
if (trail)
|
||||||
$"DELETE TOP (@n) FROM {_conn.Q("SampleModified")} WHERE TagId = @id AND SourceTimeUtc < @cutoff",
|
{
|
||||||
cutoff, id, ct).ConfigureAwait(false);
|
modified += await DeleteLoopAsync(conn,
|
||||||
|
$"DELETE TOP (@n) FROM {_conn.Q("SampleModified")} WHERE TagId = @id AND SourceTimeUtc < @cutoff",
|
||||||
|
cutoff, id, ct).ConfigureAwait(false);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var events = await DeleteLoopAsync(conn,
|
var events = await DeleteLoopAsync(conn,
|
||||||
$"DELETE TOP (@n) FROM {_conn.Q("Event")} WHERE EventTimeUtc < @cutoff",
|
$"DELETE TOP (@n) FROM {_conn.Q("Event")} WHERE EventTimeUtc < @cutoff",
|
||||||
cutoff, null, ct).ConfigureAwait(false);
|
cutoff, null, ct).ConfigureAwait(false);
|
||||||
|
|
||||||
return
|
return trail
|
||||||
[
|
?
|
||||||
new RetentionPurgeResult("Sample", samples),
|
[
|
||||||
new RetentionPurgeResult("SampleModified", modified),
|
new RetentionPurgeResult("Sample", samples),
|
||||||
new RetentionPurgeResult("Event", events),
|
new RetentionPurgeResult("SampleModified", modified),
|
||||||
];
|
new RetentionPurgeResult("Event", events),
|
||||||
|
]
|
||||||
|
:
|
||||||
|
[
|
||||||
|
new RetentionPurgeResult("Sample", samples),
|
||||||
|
new RetentionPurgeResult("Event", events),
|
||||||
|
];
|
||||||
}
|
}
|
||||||
finally
|
finally
|
||||||
{
|
{
|
||||||
@@ -137,8 +152,7 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
|||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await using var cmd = _conn.CreateCommand(conn,
|
await using var cmd = _conn.CreateCommand(conn, ReleaseSql);
|
||||||
"EXEC sp_releaseapplock @Resource = @res, @LockOwner = 'Session';");
|
|
||||||
cmd.Parameters.Add(new SqlParameter("@res", SqlDbType.NVarChar, 255) { Value = LockResource });
|
cmd.Parameters.Add(new SqlParameter("@res", SqlDbType.NVarChar, 255) { Value = LockResource });
|
||||||
await cmd.ExecuteNonQueryAsync(CancellationToken.None).ConfigureAwait(false);
|
await cmd.ExecuteNonQueryAsync(CancellationToken.None).ConfigureAwait(false);
|
||||||
}
|
}
|
||||||
|
|||||||
+30
-4
@@ -15,15 +15,27 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
|||||||
/// SQL, so a missing schema (or a SQL outage) does not serialise every write, read and alarm event behind one
|
/// SQL, so a missing schema (or a SQL outage) does not serialise every write, read and alarm event behind one
|
||||||
/// semaphore and one round-trip each.
|
/// semaphore and one round-trip each.
|
||||||
/// With <c>AutoCreateSchema=false</c> it only verifies <c>MAX(Version) >= </c><see cref="RequiredVersion"/>.
|
/// With <c>AutoCreateSchema=false</c> it only verifies <c>MAX(Version) >= </c><see cref="RequiredVersion"/>.
|
||||||
|
/// <para><b>Schema v1 is accepted (Gitea #674).</b> A database at version 1 (no <c>SampleModified</c>) still serves
|
||||||
|
/// live writes, raw / processed / at-time reads, alarm events and retention; only the Part 11 modification trail is
|
||||||
|
/// unavailable — <see cref="SupportsModifiedTrail"/> is false, modified reads and HistoryUpdate report
|
||||||
|
/// <c>BadHistoryOperationUnsupported</c> without touching SQL, and one Warning is logged per instance (= per boot: the
|
||||||
|
/// provider shares one bootstrapper). Before #674 a v1 database refused EVERY historian call, continuous-historization
|
||||||
|
/// drains included, until script 002 was applied by hand.</para>
|
||||||
/// </summary>
|
/// </summary>
|
||||||
internal sealed partial class SqlServerSchemaBootstrapper
|
internal sealed partial class SqlServerSchemaBootstrapper
|
||||||
{
|
{
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// The schema version this build requires: 2 = <c>002_sample_modified.sql</c> (Part 11 modified-value
|
/// The schema version this build requires to serve at all: 1 = <c>001_initial.sql</c>. Gitea #674 lowered it from
|
||||||
/// history, Gitea #668). An existing v1 schema is upgraded in place on the next bootstrap when
|
/// 2 — a missing modification trail disables only the features that need it (<see cref="ModifiedTrailVersion"/>).
|
||||||
/// <c>AutoCreateSchema</c> is on; otherwise the script must be applied by hand.
|
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public const int RequiredVersion = 2;
|
public const int RequiredVersion = 1;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The schema version that carries the Part 11 modification trail: 2 = <c>002_sample_modified.sql</c> (Gitea #668).
|
||||||
|
/// An auto-creating bootstrap always upgrades to it; with <c>AutoCreateSchema=false</c> the script must be applied by
|
||||||
|
/// hand (and the node restarted — the verdict is taken once per instance).
|
||||||
|
/// </summary>
|
||||||
|
public const int ModifiedTrailVersion = 2;
|
||||||
|
|
||||||
private const string LockResource = "OtOpcUaHistorianSchema";
|
private const string LockResource = "OtOpcUaHistorianSchema";
|
||||||
private const int LockTimeoutMs = 30_000;
|
private const int LockTimeoutMs = 30_000;
|
||||||
@@ -34,6 +46,13 @@ internal sealed partial class SqlServerSchemaBootstrapper
|
|||||||
private readonly SemaphoreSlim _gate = new(1, 1);
|
private readonly SemaphoreSlim _gate = new(1, 1);
|
||||||
private readonly BootstrapFailureCache _failure; // guarded by _gate
|
private readonly BootstrapFailureCache _failure; // guarded by _gate
|
||||||
private volatile bool _ready;
|
private volatile bool _ready;
|
||||||
|
private volatile bool _supportsModifiedTrail;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Whether the schema carries the Part 11 modification trail (version >= <see cref="ModifiedTrailVersion"/>).
|
||||||
|
/// Meaningful only after <see cref="EnsureAsync"/> has succeeded (false before). Taken once per instance.
|
||||||
|
/// </summary>
|
||||||
|
public bool SupportsModifiedTrail => _supportsModifiedTrail;
|
||||||
|
|
||||||
/// <summary>Creates the bootstrapper.</summary>
|
/// <summary>Creates the bootstrapper.</summary>
|
||||||
/// <param name="conn">Connection helper.</param>
|
/// <param name="conn">Connection helper.</param>
|
||||||
@@ -99,6 +118,13 @@ internal sealed partial class SqlServerSchemaBootstrapper
|
|||||||
throw new HistorianSchemaMissingException(
|
throw new HistorianSchemaMissingException(
|
||||||
$"Historian schema [{_conn.Schema}] is missing version {RequiredVersion} (found {v}) and AutoCreateSchema is false. " +
|
$"Historian schema [{_conn.Schema}] is missing version {RequiredVersion} (found {v}) and AutoCreateSchema is false. " +
|
||||||
"Apply the docs/historian/sqlserver/NNN_*.sql scripts above version " + v + " in order, or enable AutoCreateSchema.");
|
"Apply the docs/historian/sqlserver/NNN_*.sql scripts above version " + v + " in order, or enable AutoCreateSchema.");
|
||||||
|
_supportsModifiedTrail = v >= ModifiedTrailVersion;
|
||||||
|
if (!_supportsModifiedTrail)
|
||||||
|
_log.LogWarning("Historian schema [{Schema}] is at version {Version}, below {TrailVersion}, and AutoCreateSchema is false: " +
|
||||||
|
"the Part 11 modification trail is unavailable, so modified-value history reads and HistoryUpdate report " +
|
||||||
|
"BadHistoryOperationUnsupported and retention skips SampleModified. Live historization, reads, events and retention " +
|
||||||
|
"are unaffected. Apply docs/historian/sqlserver/002_sample_modified.sql and restart the node to enable them.",
|
||||||
|
_conn.Schema, v, ModifiedTrailVersion);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task ApplyAsync(SqlConnection conn, CancellationToken ct)
|
private async Task ApplyAsync(SqlConnection conn, CancellationToken ct)
|
||||||
|
|||||||
@@ -0,0 +1,28 @@
|
|||||||
|
using ZB.MOM.WW.OtOpcUa.Cluster;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Names the per-role appsettings overlay (<c>appsettings.{suffix}.json</c>) for a node.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// The suffix is built from the node's PLAIN roles only. <c>cluster-{ClusterId}</c> roles are mesh-membership
|
||||||
|
/// tags (and carry a verbatim, operator-chosen ClusterId), not configuration profiles — folding one into the
|
||||||
|
/// filename produced names like <c>appsettings.admin-cluster-MAIN-driver.json</c> that never exist, so on every
|
||||||
|
/// node with a cluster role the shipped overlays silently never loaded.
|
||||||
|
/// </remarks>
|
||||||
|
internal static class RoleOverlay
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// The overlay filename suffix: the plain (non-cluster) roles, ordinal-sorted and joined with <c>-</c>, or
|
||||||
|
/// <see langword="null"/> when the node has none.
|
||||||
|
/// </summary>
|
||||||
|
public static string? Suffix(IReadOnlyList<string> gossipedRoles)
|
||||||
|
{
|
||||||
|
var plain = gossipedRoles
|
||||||
|
.Where(r => !string.IsNullOrWhiteSpace(r) && !RoleParser.IsClusterRole(r))
|
||||||
|
.OrderBy(r => r, StringComparer.Ordinal)
|
||||||
|
.ToList();
|
||||||
|
return plain.Count == 0 ? null : string.Join('-', plain);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -252,13 +252,15 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
|||||||
if (_primaryRoleSource is { } primaryRoleSource)
|
if (_primaryRoleSource is { } primaryRoleSource)
|
||||||
{
|
{
|
||||||
_server.SetNativeAlarmAckPrimaryGate(() => primaryRoleSource.ShouldActAsPrimary);
|
_server.SetNativeAlarmAckPrimaryGate(() => primaryRoleSource.ShouldActAsPrimary);
|
||||||
|
// Gitea #674: HistoryUpdate is single-writer — only the Primary rewrites history (BadInvalidState elsewhere).
|
||||||
|
_server.SetHistoryUpdatePrimaryGate(() => primaryRoleSource.ShouldActAsPrimary);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
// Every driver node registers one (AddOtOpcUaRuntime), so this is a wiring regression, not a mode:
|
// Every driver node registers one (AddOtOpcUaRuntime), so this is a wiring regression, not a mode:
|
||||||
// say so rather than let a Secondary silently go back to accepting acks it will revert.
|
// say so rather than let a Secondary silently go back to accepting acks it will revert.
|
||||||
_logger.LogWarning(
|
_logger.LogWarning(
|
||||||
"OtOpcUaServerHostedService: no IDriverPrimaryRoleSource registered — native-alarm acknowledges are accepted on every node regardless of redundancy role");
|
"OtOpcUaServerHostedService: no IDriverPrimaryRoleSource registered — native-alarm acknowledges and HistoryUpdate are accepted on every node regardless of redundancy role");
|
||||||
}
|
}
|
||||||
|
|
||||||
// Wire the reverse-path inbound operator-write gateway: a client write to a writable equipment-tag
|
// Wire the reverse-path inbound operator-write gateway: a client write to a writable equipment-tag
|
||||||
@@ -317,6 +319,11 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
|||||||
_server.NodeManager.MaxConcurrentHistoryReadBatches = _serverHistorianOptions.MaxConcurrentHistoryReadBatches;
|
_server.NodeManager.MaxConcurrentHistoryReadBatches = _serverHistorianOptions.MaxConcurrentHistoryReadBatches;
|
||||||
_server.NodeManager.HistoryReadDeadline = _serverHistorianOptions.HistoryReadDeadline;
|
_server.NodeManager.HistoryReadDeadline = _serverHistorianOptions.HistoryReadDeadline;
|
||||||
|
|
||||||
|
// Gitea #674: HistoryUpdate refuses a SourceTimestamp beyond now + HistoryUpdateMaxFutureSkew (BadOutOfRange),
|
||||||
|
// judged against the host clock.
|
||||||
|
_server.NodeManager.HistoryUpdateMaxFutureSkew = _serverHistorianOptions.HistoryUpdateMaxFutureSkew;
|
||||||
|
_server.NodeManager.Clock = _time;
|
||||||
|
|
||||||
// ServiceLevel publisher needs IServerInternal — only available after Start.
|
// ServiceLevel publisher needs IServerInternal — only available after Start.
|
||||||
if (_server.CurrentInstance is { } serverInternal)
|
if (_server.CurrentInstance is { } serverInternal)
|
||||||
{
|
{
|
||||||
@@ -382,6 +389,7 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
|||||||
// Clear the native-alarm-ack router so a late ack doesn't Tell a stopping DriverHostActor.
|
// Clear the native-alarm-ack router so a late ack doesn't Tell a stopping DriverHostActor.
|
||||||
_server?.SetNativeAlarmAckRouter(null);
|
_server?.SetNativeAlarmAckRouter(null);
|
||||||
_server?.SetNativeAlarmAckPrimaryGate(null);
|
_server?.SetNativeAlarmAckPrimaryGate(null);
|
||||||
|
_server?.SetHistoryUpdatePrimaryGate(null);
|
||||||
// Restore the Null historian so a late HistoryRead doesn't hit a disposed read client.
|
// Restore the Null historian so a late HistoryRead doesn't hit a disposed read client.
|
||||||
_server?.SetHistorianDataSource(null);
|
_server?.SetHistorianDataSource(null);
|
||||||
_server?.SetHistorianDataUpdater(null);
|
_server?.SetHistorianDataUpdater(null);
|
||||||
|
|||||||
@@ -90,8 +90,9 @@ builder.WebHost.UseStaticWebAssets();
|
|||||||
builder.Host.UseWindowsService(options => options.ServiceName = "OtOpcUaHost");
|
builder.Host.UseWindowsService(options => options.ServiceName = "OtOpcUaHost");
|
||||||
|
|
||||||
// Per-role appsettings overlay: appsettings.{role}.json (single role) or appsettings.admin-driver.json
|
// Per-role appsettings overlay: appsettings.{role}.json (single role) or appsettings.admin-driver.json
|
||||||
// (both). Optional — base appsettings.json carries enough to boot if these don't exist.
|
// (both). Optional — base appsettings.json carries enough to boot if these don't exist. Built from the
|
||||||
var roleSuffix = roles.Count == 0 ? null : string.Join('-', roles.OrderBy(r => r, StringComparer.Ordinal));
|
// PLAIN roles only (#678.6): a cluster-{ClusterId} role in the name pointed at a file that never exists.
|
||||||
|
var roleSuffix = RoleOverlay.Suffix(roles);
|
||||||
if (roleSuffix is not null)
|
if (roleSuffix is not null)
|
||||||
{
|
{
|
||||||
builder.Configuration.AddJsonFile($"appsettings.{roleSuffix}.json", optional: true, reloadOnChange: true);
|
builder.Configuration.AddJsonFile($"appsettings.{roleSuffix}.json", optional: true, reloadOnChange: true);
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
"MinimumLevel": {
|
"MinimumLevel": {
|
||||||
"Default": "Information",
|
"Default": "Information",
|
||||||
"Override": {
|
"Override": {
|
||||||
"Opc.Ua": "Debug",
|
"Opc.Ua": "Information",
|
||||||
"Akka": "Information"
|
"Akka": "Information"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -46,6 +46,11 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
|||||||
/// <see cref="HistoryCursorKind.Modified"/> only: the normalised window start, so a resumed page keeps the original
|
/// <see cref="HistoryCursorKind.Modified"/> only: the normalised window start, so a resumed page keeps the original
|
||||||
/// window and direction (the modified read resumes by key, not by moving the window start).
|
/// window and direction (the modified read resumes by key, not by moving the window start).
|
||||||
/// </param>
|
/// </param>
|
||||||
|
/// <param name="Reverse">
|
||||||
|
/// <see cref="HistoryCursorKind.Events"/> only (Gitea #675): the read is newest-first over
|
||||||
|
/// <c>(EndUtc, NextStartUtc]</c>. A resumed events read takes its direction from here, never from the resume
|
||||||
|
/// request's details. Raw keeps deriving it from the window (<c>NextStartUtc > EndUtc</c>).
|
||||||
|
/// </param>
|
||||||
internal sealed record HistoryContinuationState(
|
internal sealed record HistoryContinuationState(
|
||||||
string Tagname,
|
string Tagname,
|
||||||
DateTime NextStartUtc,
|
DateTime NextStartUtc,
|
||||||
@@ -54,7 +59,8 @@ internal sealed record HistoryContinuationState(
|
|||||||
uint NumValuesPerNode,
|
uint NumValuesPerNode,
|
||||||
HistoryCursorKind Kind = HistoryCursorKind.Raw,
|
HistoryCursorKind Kind = HistoryCursorKind.Raw,
|
||||||
long LastModifiedId = 0,
|
long LastModifiedId = 0,
|
||||||
DateTime WindowStartUtc = default);
|
DateTime WindowStartUtc = default,
|
||||||
|
bool Reverse = false);
|
||||||
|
|
||||||
/// <summary>The HistoryRead kind that issued a <see cref="HistoryContinuationState"/>.</summary>
|
/// <summary>The HistoryRead kind that issued a <see cref="HistoryContinuationState"/>.</summary>
|
||||||
internal enum HistoryCursorKind
|
internal enum HistoryCursorKind
|
||||||
|
|||||||
@@ -102,7 +102,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
/// <see cref="_alarmConditions"/> / <see cref="_notifierFolders"/>.</summary>
|
/// <see cref="_alarmConditions"/> / <see cref="_notifierFolders"/>.</summary>
|
||||||
private readonly Dictionary<RealmKey, AlarmNotifierWiring> _alarmNotifierWiring = new();
|
private readonly Dictionary<RealmKey, AlarmNotifierWiring> _alarmNotifierWiring = new();
|
||||||
/// <summary>Phase C: event-notifier folder NodeId-identifier → the event-history source
|
/// <summary>Phase C: event-notifier folder NodeId-identifier → the event-history source
|
||||||
/// name passed to <see cref="IHistorianDataSource.ReadEventsAsync"/>. The equipment-folder NodeId
|
/// name passed to <see cref="IHistorianDataSource.ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/>. The equipment-folder NodeId
|
||||||
/// identifier IS the equipment id, which IS the sourceName, so key and value are the same string;
|
/// identifier IS the equipment id, which IS the sourceName, so key and value are the same string;
|
||||||
/// the map's presence (not its value) is what makes a folder an event-history source. Populated by
|
/// the map's presence (not its value) is what makes a folder an event-history source. Populated by
|
||||||
/// <see cref="EnsureFolderIsEventNotifier"/> only when a real historian is wired at promotion time,
|
/// <see cref="EnsureFolderIsEventNotifier"/> only when a real historian is wired at promotion time,
|
||||||
@@ -247,6 +247,48 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public Func<bool>? NativeAlarmAckPrimaryGate { get; set; }
|
public Func<bool>? NativeAlarmAckPrimaryGate { get; set; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674 — whether this node may serve an OPC UA HistoryUpdate right now: the redundancy data-plane
|
||||||
|
/// verdict (<c>PrimaryGatePolicy.ShouldServiceAsPrimary</c>, as published to <c>IDriverPrimaryRoleSource</c>) —
|
||||||
|
/// the same source as <see cref="NativeAlarmAckPrimaryGate"/>.
|
||||||
|
/// <para>
|
||||||
|
/// <c>false</c> ⇒ every node of the request that passed authorization is refused <c>BadInvalidState</c>
|
||||||
|
/// and nothing reaches the updater. Rewriting history is a single-writer operation: the SQL backends'
|
||||||
|
/// prior-value capture (and QuestDB's per-process per-tag lock) assume one node serialises a tag's updates,
|
||||||
|
/// so both pair nodes accepting updates concurrently could record a wrong prior value in the modification
|
||||||
|
/// trail. HistoryRead is NOT gated — reads are served from any node.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// Evaluated AFTER authorization (a caller who may not update learns nothing about the node's role). It must
|
||||||
|
/// be a cheap, non-blocking read (the host passes a volatile-backed property). Null (the default) allows — the
|
||||||
|
/// single-node posture, the same default <c>UnmanagedDriverPrimaryRoleSource</c> answers with.
|
||||||
|
/// </para>
|
||||||
|
/// </summary>
|
||||||
|
public Func<bool>? HistoryUpdatePrimaryGate { get; set; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674 — how far beyond <see cref="Clock"/>'s now a HistoryUpdate (Insert / Replace / Update) item's
|
||||||
|
/// <c>SourceTimestamp</c> may lie. An item past <c>now + skew</c> is refused <c>BadOutOfRange</c> before it
|
||||||
|
/// reaches any backend: a far-future row (a 2062 typo) is never a legitimate history edit, and on a backend whose
|
||||||
|
/// retention is judged against the newest timestamp it could age out every older partition. Mirrors
|
||||||
|
/// <c>ServerHistorianOptions.HistoryUpdateMaxFutureSkew</c>; the Host sets it at <c>StartAsync</c>. Non-positive
|
||||||
|
/// disables the bound. Default 1 day. Live recorder writes are not bounded by this.
|
||||||
|
/// </summary>
|
||||||
|
public TimeSpan HistoryUpdateMaxFutureSkew { get; set; } = TimeSpan.FromDays(1);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The clock the HistoryUpdate future-skew bound (<see cref="HistoryUpdateMaxFutureSkew"/>) reads. The Host
|
||||||
|
/// assigns its <see cref="TimeProvider"/> at <c>StartAsync</c>; tests inject a fixed clock. Defaults to
|
||||||
|
/// <see cref="TimeProvider.System"/>; assigning null restores it.
|
||||||
|
/// </summary>
|
||||||
|
public TimeProvider Clock
|
||||||
|
{
|
||||||
|
get => _clock;
|
||||||
|
set => _clock = value ?? TimeProvider.System;
|
||||||
|
}
|
||||||
|
|
||||||
|
private volatile TimeProvider _clock = TimeProvider.System;
|
||||||
|
|
||||||
private volatile IOpcUaNodeWriteGateway _nodeWriteGateway = NullOpcUaNodeWriteGateway.Instance;
|
private volatile IOpcUaNodeWriteGateway _nodeWriteGateway = NullOpcUaNodeWriteGateway.Instance;
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -5161,6 +5203,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
var endUtc = details.EndTime;
|
var endUtc = details.EndTime;
|
||||||
var numValuesPerNode = details.NumValuesPerNode;
|
var numValuesPerNode = details.NumValuesPerNode;
|
||||||
var boundarySkip = 0;
|
var boundarySkip = 0;
|
||||||
|
bool reverse;
|
||||||
|
|
||||||
if (resumed)
|
if (resumed)
|
||||||
{
|
{
|
||||||
@@ -5180,36 +5223,59 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
endUtc = state.EndUtc;
|
endUtc = state.EndUtc;
|
||||||
boundarySkip = state.BoundarySkipCount;
|
boundarySkip = state.BoundarySkipCount;
|
||||||
numValuesPerNode = state.NumValuesPerNode;
|
numValuesPerNode = state.NumValuesPerNode;
|
||||||
|
// The direction is the CURSOR's (Gitea #675), never re-derived from the resume request's details: a
|
||||||
|
// reverse read resumes reverse, a forward one forward.
|
||||||
|
reverse = state.Reverse;
|
||||||
}
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
// Part 11: with one bound unset, an unset EndTime (MinValue) with a StartTime means "page FORWARD from
|
||||||
|
// StartTime" — normalise it to an open upper bound, or the cursor would store MinValue and the read would
|
||||||
|
// look reversed. An unset StartTime with an EndTime and a count cap means "page BACKWARD from EndTime"
|
||||||
|
// (#675, mirroring raw): the reverse window (MinValue, EndTime]. Without a count cap an end-only read
|
||||||
|
// keeps its historical forward [MinValue, EndTime) meaning.
|
||||||
|
if (endUtc == DateTime.MinValue && startUtc != DateTime.MinValue)
|
||||||
|
endUtc = new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc); // Utc kind: a proto Timestamp conversion rejects Unspecified
|
||||||
|
else if (numValuesPerNode > 0 && startUtc == DateTime.MinValue && endUtc != DateTime.MinValue)
|
||||||
|
(startUtc, endUtc) = (endUtc, new DateTime(0, DateTimeKind.Utc));
|
||||||
|
|
||||||
// Part 11: an unset EndTime (MinValue) with a StartTime means "page forward from StartTime". Normalise it
|
// Part 11: StartTime later than EndTime is a reverse read — events newest-first over (end, start].
|
||||||
// to an open upper bound, or the cursor would store MinValue and a backend would swap the reversed
|
reverse = startUtc > endUtc;
|
||||||
// resume window. A resumed read is always forward (the cursor is only ever stored for a forward read),
|
}
|
||||||
// so its direction comes from the cursor, not from the resume request's details.
|
|
||||||
if (!resumed && endUtc == DateTime.MinValue && startUtc != DateTime.MinValue)
|
|
||||||
endUtc = new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc); // Utc kind: a proto Timestamp conversion rejects Unspecified
|
|
||||||
// Reverse event reads (start > end) are not paged: the cursor below is forward-only.
|
|
||||||
var forward = resumed || startUtc <= endUtc;
|
|
||||||
|
|
||||||
// NOT under the node-manager Lock — awaiting the async source is safe here.
|
// NOT under the node-manager Lock — awaiting the async source is safe here.
|
||||||
// NumValuesPerNode==0 means "no limit — return ALL values" per OPC UA Part 4/11, so it is translated
|
// NumValuesPerNode==0 means "no limit — return ALL values" per OPC UA Part 4/11, so it is translated
|
||||||
// to UNBOUNDED (EventMaxEvents) rather than the backend's <=0 default-cap sentinel.
|
// to UNBOUNDED (EventMaxEvents) rather than the backend's <=0 default-cap sentinel. A resumed page re-reads its
|
||||||
|
// boundary timestamp inclusively and then drops the `boundarySkip` ties already emitted, so it asks for that
|
||||||
|
// many more rows than the cap — otherwise every resumed page would come back short by the skip.
|
||||||
|
var fetch = EventMaxEvents(numValuesPerNode);
|
||||||
|
if (numValuesPerNode > 0 && boundarySkip > 0)
|
||||||
|
fetch = (int)Math.Min(int.MaxValue - 1L, (long)fetch + boundarySkip);
|
||||||
var sourceResult = await _historianDataSource.ReadEventsAsync(
|
var sourceResult = await _historianDataSource.ReadEventsAsync(
|
||||||
sourceName, startUtc, endUtc, EventMaxEvents(numValuesPerNode), ct).ConfigureAwait(false);
|
sourceName, startUtc, endUtc, fetch, reverse, ct).ConfigureAwait(false);
|
||||||
|
|
||||||
// Truncation is the SOURCE's word (non-null continuation point) — never inferred from the count, so a
|
// Truncation is the SOURCE's word (non-null continuation point) — never inferred from the count, so a
|
||||||
// page that happens to be exactly cap-sized but complete never emits a point.
|
// page that happens to be exactly cap-sized but complete never emits a point.
|
||||||
var truncated = sourceResult.ContinuationPoint is not null && numValuesPerNode > 0 && forward;
|
var truncated = sourceResult.ContinuationPoint is not null && numValuesPerNode > 0;
|
||||||
|
|
||||||
IReadOnlyList<HistoricalEvent> events = sourceResult.Events;
|
IReadOnlyList<HistoricalEvent> events = sourceResult.Events;
|
||||||
if (resumed && boundarySkip > 0)
|
if (resumed && boundarySkip > 0)
|
||||||
{
|
{
|
||||||
// Drop the events at the boundary timestamp already emitted on the prior page.
|
// Drop the events at the boundary timestamp already emitted on the prior page. The resumed read is in
|
||||||
|
// the cursor's direction, so in both directions those ties lead the page.
|
||||||
var i = 0;
|
var i = 0;
|
||||||
while (i < events.Count && i < boundarySkip && events[i].EventTimeUtc == startUtc) i++;
|
while (i < events.Count && i < boundarySkip && events[i].EventTimeUtc == startUtc) i++;
|
||||||
if (i > 0) events = events.Skip(i).ToList();
|
if (i > 0) events = events.Skip(i).ToList();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The skip over-fetch can leave more than the cap (fewer boundary ties came back than were skipped — pruned
|
||||||
|
// between pages): keep the cap, and the dropped tail means the read continues.
|
||||||
|
if (numValuesPerNode > 0 && events.Count > numValuesPerNode)
|
||||||
|
{
|
||||||
|
events = events.Take((int)Math.Min(int.MaxValue, numValuesPerNode)).ToList();
|
||||||
|
truncated = true;
|
||||||
|
}
|
||||||
|
|
||||||
// The cursor to store, if any. Saved only AFTER the page is projected, so a projection throw can
|
// The cursor to store, if any. Saved only AFTER the page is projected, so a projection throw can
|
||||||
// never orphan a stored cursor.
|
// never orphan a stored cursor.
|
||||||
HistoryContinuationState? pendingCursor = null;
|
HistoryContinuationState? pendingCursor = null;
|
||||||
@@ -5222,10 +5288,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
// with an explicit bounded cap and page WITHIN it (same scheme as raw).
|
// with an explicit bounded cap and page WITHIN it (same scheme as raw).
|
||||||
var resolution = Math.Max(1L, _historianDataSource.TimestampResolutionTicks);
|
var resolution = Math.Max(1L, _historianDataSource.TimestampResolutionTicks);
|
||||||
var overfetchCap = (int)Math.Min(int.MaxValue - 1L, (long)Math.Max(1, MaxTieClusterOverfetch) + 1);
|
var overfetchCap = (int)Math.Min(int.MaxValue - 1L, (long)Math.Max(1, MaxTieClusterOverfetch) + 1);
|
||||||
var cluster = (await _historianDataSource.ReadEventsAsync(
|
IReadOnlyList<HistoricalEvent> cluster = (await _historianDataSource.ReadEventsAsync(
|
||||||
sourceName, startUtc, startUtc.AddTicks(resolution), overfetchCap, ct)
|
sourceName, startUtc, startUtc.AddTicks(resolution), overfetchCap, reverse: false, ct)
|
||||||
.ConfigureAwait(false)).Events;
|
.ConfigureAwait(false)).Events;
|
||||||
|
|
||||||
|
// The over-fetch reads the cluster FORWARD (ascending EventId), but a reverse read emitted its ties — and
|
||||||
|
// counted boundarySkip — in descending order. Slice in the read's own direction, as raw does.
|
||||||
|
if (reverse) cluster = cluster.Reverse().ToList();
|
||||||
|
|
||||||
if (cluster.Count > MaxTieClusterOverfetch)
|
if (cluster.Count > MaxTieClusterOverfetch)
|
||||||
{
|
{
|
||||||
_logger.LogError(
|
_logger.LogError(
|
||||||
@@ -5240,7 +5310,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
HistoryPaging.SliceTieCluster(
|
HistoryPaging.SliceTieCluster(
|
||||||
cluster.Count, boundarySkip, numValuesPerNode, startUtc, endUtc,
|
cluster.Count, boundarySkip, numValuesPerNode, startUtc, endUtc,
|
||||||
out var sliceStart, out var sliceCount, out var nextStartUtc, out var nextSkip,
|
out var sliceStart, out var sliceCount, out var nextStartUtc, out var nextSkip,
|
||||||
reverse: false, resolutionTicks: resolution);
|
reverse: reverse, resolutionTicks: resolution);
|
||||||
|
|
||||||
var slice = new List<HistoricalEvent>(sliceCount);
|
var slice = new List<HistoricalEvent>(sliceCount);
|
||||||
for (var i = sliceStart; i < sliceStart + sliceCount; i++) slice.Add(cluster[i]);
|
for (var i = sliceStart; i < sliceStart + sliceCount; i++) slice.Add(cluster[i]);
|
||||||
@@ -5249,17 +5319,20 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
if (nextStartUtc is { } resumeAt)
|
if (nextStartUtc is { } resumeAt)
|
||||||
{
|
{
|
||||||
pendingCursor = new HistoryContinuationState(
|
pendingCursor = new HistoryContinuationState(
|
||||||
sourceName ?? string.Empty, resumeAt, endUtc, nextSkip, numValuesPerNode, HistoryCursorKind.Events);
|
sourceName ?? string.Empty, resumeAt, endUtc, nextSkip, numValuesPerNode, HistoryCursorKind.Events,
|
||||||
|
Reverse: reverse);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
else if (truncated && events.Count > 0)
|
else if (truncated && events.Count > 0)
|
||||||
{
|
{
|
||||||
// Cursor from the last event: resume at its time INCLUSIVE, skipping the trailing ties emitted.
|
// Cursor from the last event: resume at its time INCLUSIVE, skipping the trailing ties emitted. In a
|
||||||
|
// reverse read the last event is the OLDEST on the page, and the resumed (end, T] read starts there.
|
||||||
var boundary = events[^1].EventTimeUtc;
|
var boundary = events[^1].EventTimeUtc;
|
||||||
var skip = 0;
|
var skip = 0;
|
||||||
for (var i = events.Count - 1; i >= 0 && events[i].EventTimeUtc == boundary; i--) skip++;
|
for (var i = events.Count - 1; i >= 0 && events[i].EventTimeUtc == boundary; i--) skip++;
|
||||||
pendingCursor = new HistoryContinuationState(
|
pendingCursor = new HistoryContinuationState(
|
||||||
sourceName ?? string.Empty, boundary, endUtc, skip, numValuesPerNode, HistoryCursorKind.Events);
|
sourceName ?? string.Empty, boundary, endUtc, skip, numValuesPerNode, HistoryCursorKind.Events,
|
||||||
|
Reverse: reverse);
|
||||||
}
|
}
|
||||||
|
|
||||||
var historyEvent = ProjectEvents(events, selectClauses);
|
var historyEvent = ProjectEvents(events, selectClauses);
|
||||||
@@ -5280,6 +5353,15 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
errors[handle.Index] = StatusCodes.BadTimeout;
|
errors[handle.Index] = StatusCodes.BadTimeout;
|
||||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
||||||
}
|
}
|
||||||
|
catch (NotSupportedException ex)
|
||||||
|
{
|
||||||
|
// The backend cannot serve this read's shape — today a reverse (newest-first) read on Gateway / Null
|
||||||
|
// (#675). A client request the backend refuses, not a backend fault: Warning, not Error.
|
||||||
|
_logger.LogWarning("OtOpcUaNodeManager: HistoryReadEvents for node {NodeId} is not supported by the historian backend: {Reason}",
|
||||||
|
handle.NodeId, ex.Message);
|
||||||
|
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||||
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
// One node's backend failure must not throw out of the batch — surface Bad for THIS node
|
// One node's backend failure must not throw out of the batch — surface Bad for THIS node
|
||||||
@@ -5373,6 +5455,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
List<NodeHandle> nodesToProcess,
|
List<NodeHandle> nodesToProcess,
|
||||||
IDictionary<NodeId, NodeState> cache)
|
IDictionary<NodeId, NodeState> cache)
|
||||||
{
|
{
|
||||||
|
// #674: one "now" per request, so every item of the batch is judged against the same bound.
|
||||||
|
var latest = LatestAcceptedSourceTime();
|
||||||
ServeHistoryUpdate(context, nodesToProcess, results, errors, handle =>
|
ServeHistoryUpdate(context, nodesToProcess, results, errors, handle =>
|
||||||
{
|
{
|
||||||
var details = nodesToUpdate[handle.Index];
|
var details = nodesToUpdate[handle.Index];
|
||||||
@@ -5386,7 +5470,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
}
|
}
|
||||||
|
|
||||||
var items = (IList<DataValue>?)details.UpdateValues ?? Array.Empty<DataValue>();
|
var items = (IList<DataValue>?)details.UpdateValues ?? Array.Empty<DataValue>();
|
||||||
return (updater, tagname, who, ct) => UpdateDataAsync(updater, tagname, mode, items, who, ct);
|
return (updater, tagname, who, ct) => UpdateDataAsync(updater, tagname, mode, items, latest, who, ct);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -5448,22 +5532,40 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: the latest <c>SourceTimestamp</c> a HistoryUpdate item may carry — <see cref="Clock"/>'s now plus
|
||||||
|
/// <see cref="HistoryUpdateMaxFutureSkew"/> — or <c>null</c> when the bound is disabled (non-positive skew).
|
||||||
|
/// Saturates at <see cref="DateTime.MaxValue"/> rather than overflowing for an absurd configured skew.
|
||||||
|
/// </summary>
|
||||||
|
/// <returns>The inclusive upper bound, or null for unbounded.</returns>
|
||||||
|
private DateTime? LatestAcceptedSourceTime()
|
||||||
|
{
|
||||||
|
var skew = HistoryUpdateMaxFutureSkew;
|
||||||
|
if (skew <= TimeSpan.Zero) return null;
|
||||||
|
var now = _clock.GetUtcNow().UtcDateTime;
|
||||||
|
return skew.Ticks > DateTime.MaxValue.Ticks - now.Ticks
|
||||||
|
? new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc)
|
||||||
|
: now + skew;
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Applies one node's UpdateDataDetails: items the server can refuse on its own
|
/// Applies one node's UpdateDataDetails: items the server can refuse on its own
|
||||||
/// (<c>SourceTimestamp == DateTime.MinValue</c> ⇒ <c>BadInvalidTimestamp</c>; a value
|
/// (<c>SourceTimestamp == DateTime.MinValue</c> ⇒ <c>BadInvalidTimestamp</c>; a <c>SourceTimestamp</c> later than
|
||||||
/// <see cref="HistorianValue.TryFromObject"/> cannot type ⇒ <c>BadTypeMismatch</c>) are answered locally,
|
/// <paramref name="latest"/> ⇒ <c>BadOutOfRange</c>, #674; a value <see cref="HistorianValue.TryFromObject"/>
|
||||||
/// the rest go to the updater in ONE call, and the per-item codes are stitched back in request order.
|
/// cannot type ⇒ <c>BadTypeMismatch</c>) are answered locally, the rest go to the updater in ONE call, and the
|
||||||
|
/// per-item codes are stitched back in request order.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
/// <param name="updater">The wired updater.</param>
|
/// <param name="updater">The wired updater.</param>
|
||||||
/// <param name="tagname">The node's historian tagname.</param>
|
/// <param name="tagname">The node's historian tagname.</param>
|
||||||
/// <param name="mode">The update mode.</param>
|
/// <param name="mode">The update mode.</param>
|
||||||
/// <param name="items">The request's values, in order.</param>
|
/// <param name="items">The request's values, in order.</param>
|
||||||
|
/// <param name="latest">The latest accepted source time (<see cref="LatestAcceptedSourceTime"/>); null = unbounded.</param>
|
||||||
/// <param name="who">The caller, recorded in the modification trail.</param>
|
/// <param name="who">The caller, recorded in the modification trail.</param>
|
||||||
/// <param name="ct">The request deadline.</param>
|
/// <param name="ct">The request deadline.</param>
|
||||||
/// <returns>The node's result, carrying one operation result per item.</returns>
|
/// <returns>The node's result, carrying one operation result per item.</returns>
|
||||||
private static async Task<HistoryUpdateResult> UpdateDataAsync(
|
private static async Task<HistoryUpdateResult> UpdateDataAsync(
|
||||||
IHistorianDataUpdater updater, string tagname, HistoryUpdateMode mode, IList<DataValue> items,
|
IHistorianDataUpdater updater, string tagname, HistoryUpdateMode mode, IList<DataValue> items,
|
||||||
HistoryUpdateContext who, CancellationToken ct)
|
DateTime? latest, HistoryUpdateContext who, CancellationToken ct)
|
||||||
{
|
{
|
||||||
var codes = new uint[items.Count];
|
var codes = new uint[items.Count];
|
||||||
var toSend = new List<HistorizationValue>(items.Count);
|
var toSend = new List<HistorizationValue>(items.Count);
|
||||||
@@ -5479,6 +5581,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
{
|
{
|
||||||
codes[i] = StatusCodes.BadInvalidTimestamp;
|
codes[i] = StatusCodes.BadInvalidTimestamp;
|
||||||
}
|
}
|
||||||
|
else if (latest is { } bound && item.SourceTimestamp > bound)
|
||||||
|
{
|
||||||
|
// #674: a far-future history edit is refused before any backend sees it (wire timestamps are UTC).
|
||||||
|
codes[i] = StatusCodes.BadOutOfRange;
|
||||||
|
}
|
||||||
else if (!HistorianValue.TryFromObject(item.Value, out var value))
|
else if (!HistorianValue.TryFromObject(item.Value, out var value))
|
||||||
{
|
{
|
||||||
codes[i] = StatusCodes.BadTypeMismatch;
|
codes[i] = StatusCodes.BadTypeMismatch;
|
||||||
@@ -5529,6 +5636,21 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
nodesToProcess = AuthorizeHistoryUpdate(context, nodesToProcess, results, errors);
|
nodesToProcess = AuthorizeHistoryUpdate(context, nodesToProcess, results, errors);
|
||||||
if (nodesToProcess.Count == 0) return;
|
if (nodesToProcess.Count == 0) return;
|
||||||
|
|
||||||
|
// #674 — PRIMARY GATE, after authorization (a caller who may not update learns nothing about the node's role).
|
||||||
|
// Off-Primary every authorized node is refused BadInvalidState and nothing reaches the updater.
|
||||||
|
if (HistoryUpdatePrimaryGate is { } primary && !primary())
|
||||||
|
{
|
||||||
|
var refusal = new ServiceResult(StatusCodes.BadInvalidState,
|
||||||
|
new LocalizedText("This server is not the redundancy Primary; update history on the Primary (the server with the higher ServiceLevel)."));
|
||||||
|
foreach (var handle in nodesToProcess)
|
||||||
|
{
|
||||||
|
errors[handle.Index] = refusal;
|
||||||
|
results[handle.Index] = new HistoryUpdateResult { StatusCode = StatusCodes.BadInvalidState };
|
||||||
|
}
|
||||||
|
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
// Who is rewriting history — recorded on every modification-trail row (Part 11 ModificationInfo.UserName, #668).
|
// Who is rewriting history — recorded on every modification-trail row (Part 11 ModificationInfo.UserName, #668).
|
||||||
var who = HistoryUpdateCallerOf(context);
|
var who = HistoryUpdateCallerOf(context);
|
||||||
|
|
||||||
@@ -5619,7 +5741,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
private static int ClampToInt(uint value) => value > int.MaxValue ? int.MaxValue : (int)value;
|
private static int ClampToInt(uint value) => value > int.MaxValue ? int.MaxValue : (int)value;
|
||||||
|
|
||||||
/// <summary>Map a HistoryRead-Events <c>NumValuesPerNode</c> request cap onto the
|
/// <summary>Map a HistoryRead-Events <c>NumValuesPerNode</c> request cap onto the
|
||||||
/// <see cref="IHistorianDataSource.ReadEventsAsync"/> <c>maxEvents</c> argument, honouring the OPC UA
|
/// <see cref="IHistorianDataSource.ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/> <c>maxEvents</c> argument, honouring the OPC UA
|
||||||
/// Part 4/11 semantics that <c>NumValuesPerNode == 0</c> means "no limit — return ALL values".
|
/// Part 4/11 semantics that <c>NumValuesPerNode == 0</c> means "no limit — return ALL values".
|
||||||
/// We translate 0 to UNBOUNDED (<see cref="int.MaxValue"/>) — a very large positive cap — rather than
|
/// We translate 0 to UNBOUNDED (<see cref="int.MaxValue"/>) — a very large positive cap — rather than
|
||||||
/// the backend's <c>maxEvents <= 0</c> "use the default cap" sentinel, so a client asking for the
|
/// the backend's <c>maxEvents <= 0</c> "use the default cap" sentinel, so a client asking for the
|
||||||
|
|||||||
@@ -83,6 +83,21 @@ public sealed class OtOpcUaSdkServer : StandardServer
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674 — wire the redundancy verdict that decides whether this node serves an OPC UA HistoryUpdate
|
||||||
|
/// (<see cref="OtOpcUaNodeManager.HistoryUpdatePrimaryGate"/>); on any other node every authorized node of a
|
||||||
|
/// HistoryUpdate is refused <c>BadInvalidState</c>. Passing <c>null</c> clears it (allow). No-op (returns
|
||||||
|
/// <c>false</c>) when the node manager has not been created yet (mirrors <see cref="SetNativeAlarmAckPrimaryGate"/>).
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="isPrimary">A cheap, non-blocking read of the data-plane Primary verdict; null to allow.</param>
|
||||||
|
/// <returns><c>true</c> when set on a live node manager; <c>false</c> when no node manager exists yet.</returns>
|
||||||
|
public bool SetHistoryUpdatePrimaryGate(Func<bool>? isPrimary)
|
||||||
|
{
|
||||||
|
if (_otOpcUaNodeManager is null) return false;
|
||||||
|
_otOpcUaNodeManager.HistoryUpdatePrimaryGate = isPrimary;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Wire the reverse-path gateway for inbound operator writes to writable equipment-tag nodes onto the
|
/// Wire the reverse-path gateway for inbound operator writes to writable equipment-tag nodes onto the
|
||||||
/// created <see cref="OtOpcUaNodeManager"/>. The host calls this after start with an
|
/// created <see cref="OtOpcUaNodeManager"/>. The host calls this after start with an
|
||||||
|
|||||||
@@ -122,6 +122,16 @@ public sealed class ServerHistorianOptions
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public int MaxTimeWeightedRawRows { get; init; } = 1_000_000;
|
public int MaxTimeWeightedRawRows { get; init; } = 1_000_000;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674: how far into the future an OPC UA HistoryUpdate (Insert / Replace / Update) item's
|
||||||
|
/// <c>SourceTimestamp</c> may lie, measured from the server clock. An item beyond <c>now + skew</c> is refused
|
||||||
|
/// <c>BadOutOfRange</c> by the node manager before any backend sees it — a far-future row is never a legitimate
|
||||||
|
/// history edit, and on a backend that judges retention against the newest timestamp it could age out every
|
||||||
|
/// older partition. Applies to every backend. The live recorder is not bounded. Non-positive disables the bound.
|
||||||
|
/// Default 1 day.
|
||||||
|
/// </summary>
|
||||||
|
public TimeSpan HistoryUpdateMaxFutureSkew { get; init; } = TimeSpan.FromDays(1);
|
||||||
|
|
||||||
/// <summary>Returns operator-facing misconfiguration warnings for an <c>Enabled</c> historian
|
/// <summary>Returns operator-facing misconfiguration warnings for an <c>Enabled</c> historian
|
||||||
/// (empty when disabled or correctly configured). Pure — the registration logs each entry.
|
/// (empty when disabled or correctly configured). Pure — the registration logs each entry.
|
||||||
/// <para>
|
/// <para>
|
||||||
|
|||||||
@@ -108,6 +108,9 @@ public sealed class FakeOpcUaClientService : IOpcUaClientService
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public Dictionary<string, IReadOnlyList<BrowseResult>> BrowseResultsByParent { get; } = new();
|
public Dictionary<string, IReadOnlyList<BrowseResult>> BrowseResultsByParent { get; } = new();
|
||||||
|
|
||||||
|
/// <summary>Gets or sets an exception thrown by HistoryReadRawAsync (simulates a refused read).</summary>
|
||||||
|
public Exception? HistoryReadException { get; set; }
|
||||||
|
|
||||||
/// <summary>Gets or sets the history read result returned by HistoryReadRawAsync and HistoryReadAggregateAsync.</summary>
|
/// <summary>Gets or sets the history read result returned by HistoryReadRawAsync and HistoryReadAggregateAsync.</summary>
|
||||||
public IReadOnlyList<DataValue> HistoryReadResult { get; set; } = new List<DataValue>
|
public IReadOnlyList<DataValue> HistoryReadResult { get; set; } = new List<DataValue>
|
||||||
{
|
{
|
||||||
@@ -323,6 +326,7 @@ public sealed class FakeOpcUaClientService : IOpcUaClientService
|
|||||||
NodeId nodeId, DateTime startTime, DateTime endTime, int maxValues = 1000, CancellationToken ct = default)
|
NodeId nodeId, DateTime startTime, DateTime endTime, int maxValues = 1000, CancellationToken ct = default)
|
||||||
{
|
{
|
||||||
HistoryReadRawCalls.Add((nodeId, startTime, endTime, maxValues));
|
HistoryReadRawCalls.Add((nodeId, startTime, endTime, maxValues));
|
||||||
|
if (HistoryReadException != null) throw HistoryReadException;
|
||||||
return Task.FromResult(HistoryReadResult);
|
return Task.FromResult(HistoryReadResult);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -197,4 +197,53 @@ public class HistoryReadCommandTests
|
|||||||
using var console = TestConsoleHelper.CreateConsole();
|
using var console = TestConsoleHelper.CreateConsole();
|
||||||
await Should.ThrowAsync<CliFx.Exceptions.CommandException>(async () => await command.ExecuteAsync(console));
|
await Should.ThrowAsync<CliFx.Exceptions.CommandException>(async () => await command.ExecuteAsync(console));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>#677.5: the new aggregate names parse (and the aliases).</summary>
|
||||||
|
[Theory]
|
||||||
|
[InlineData("TimeAverage", AggregateType.TimeAverage)]
|
||||||
|
[InlineData("timeaverage2", AggregateType.TimeAverage2)]
|
||||||
|
[InlineData("Total", AggregateType.Total)]
|
||||||
|
[InlineData("total2", AggregateType.Total2)]
|
||||||
|
[InlineData("Interpolative", AggregateType.Interpolative)]
|
||||||
|
[InlineData("interp", AggregateType.Interpolative)]
|
||||||
|
public async Task Execute_NewAggregates_Parse(string name, AggregateType expected)
|
||||||
|
{
|
||||||
|
var fakeService = new FakeOpcUaClientService();
|
||||||
|
var command = new HistoryReadCommand(new FakeOpcUaClientServiceFactory(fakeService))
|
||||||
|
{
|
||||||
|
Url = "opc.tcp://localhost:4840",
|
||||||
|
NodeId = "ns=2;s=HistNode",
|
||||||
|
Aggregate = name,
|
||||||
|
};
|
||||||
|
|
||||||
|
using var console = TestConsoleHelper.CreateConsole();
|
||||||
|
await command.ExecuteAsync(console);
|
||||||
|
|
||||||
|
fakeService.HistoryReadAggregateCalls.ShouldHaveSingleItem().Aggregate.ShouldBe(expected);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>#674.6: a refused read is a one-line CommandException (non-zero exit), not a stack trace.</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Execute_RefusedRead_ThrowsCommandException_WithOneLineMessage()
|
||||||
|
{
|
||||||
|
var fakeService = new FakeOpcUaClientService
|
||||||
|
{
|
||||||
|
HistoryReadException = new ServiceResultException(StatusCodes.BadHistoryOperationUnsupported,
|
||||||
|
"HistoryRead of ns=2;s=HistNode failed: BadHistoryOperationUnsupported (0x80B40000)."),
|
||||||
|
};
|
||||||
|
var command = new HistoryReadCommand(new FakeOpcUaClientServiceFactory(fakeService))
|
||||||
|
{
|
||||||
|
Url = "opc.tcp://localhost:4840",
|
||||||
|
NodeId = "ns=2;s=HistNode",
|
||||||
|
};
|
||||||
|
|
||||||
|
using var console = TestConsoleHelper.CreateConsole();
|
||||||
|
var ex = await Should.ThrowAsync<CliFx.Exceptions.CommandException>(async () => await command.ExecuteAsync(console));
|
||||||
|
|
||||||
|
ex.Message.ShouldContain("BadHistoryOperationUnsupported");
|
||||||
|
ex.Message.ShouldNotContain("\n");
|
||||||
|
ex.Message.ShouldNotContain(" at ");
|
||||||
|
ex.ExitCode.ShouldNotBe(0);
|
||||||
|
fakeService.DisconnectCalled.ShouldBeTrue();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -160,4 +160,67 @@ public class EndpointSelectorTests
|
|||||||
Should.Throw<ArgumentException>(() =>
|
Should.Throw<ArgumentException>(() =>
|
||||||
EndpointSelector.SelectBest(Array.Empty<EndpointDescription>(), "", MessageSecurityMode.None));
|
EndpointSelector.SelectBest(Array.Empty<EndpointDescription>(), "", MessageSecurityMode.None));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>#678.5: a server advertising its internal port must not redirect the session off the mapped port.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void SelectBest_RewritesHostAndPort_WhenSchemeMatches()
|
||||||
|
{
|
||||||
|
var endpoints = new[] { Ep(MessageSecurityMode.None, SecurityPolicies.None, "opc.tcp://internal-host:4840/OtOpcUa") };
|
||||||
|
|
||||||
|
var best = EndpointSelector.SelectBest(endpoints, "opc.tcp://localhost:4841", MessageSecurityMode.None);
|
||||||
|
|
||||||
|
var uri = new Uri(best.EndpointUrl);
|
||||||
|
uri.Host.ShouldBe("localhost");
|
||||||
|
uri.Port.ShouldBe(4841);
|
||||||
|
uri.AbsolutePath.ShouldBe("/OtOpcUa");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A different scheme (e.g. https endpoint advertised to an opc.tcp caller) is left untouched.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void SelectBest_LeavesEndpointUntouched_WhenSchemeDiffers()
|
||||||
|
{
|
||||||
|
var endpoints = new[] { Ep(MessageSecurityMode.None, SecurityPolicies.None, "https://internal-host:4843/x") };
|
||||||
|
|
||||||
|
var best = EndpointSelector.SelectBest(endpoints, "opc.tcp://localhost:4841", MessageSecurityMode.None);
|
||||||
|
|
||||||
|
new Uri(best.EndpointUrl).Host.ShouldBe("internal-host");
|
||||||
|
new Uri(best.EndpointUrl).Port.ShouldBe(4843);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Identical host, port and scheme: nothing to rewrite.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void SelectBest_Unchanged_WhenHostPortAndSchemeMatch()
|
||||||
|
{
|
||||||
|
var endpoints = new[] { Ep(MessageSecurityMode.None, SecurityPolicies.None, "opc.tcp://server:4840") };
|
||||||
|
|
||||||
|
var best = EndpointSelector.SelectBest(endpoints, "opc.tcp://server:4840", MessageSecurityMode.None);
|
||||||
|
|
||||||
|
best.EndpointUrl.ShouldBe("opc.tcp://server:4840");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A URL typed without a port keeps the advertised port (opc.tcp has no default; Uri.Port is -1).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void SelectBest_KeepsAdvertisedPort_WhenRequestedUrlHasNoPort()
|
||||||
|
{
|
||||||
|
var endpoints = new[] { Ep(MessageSecurityMode.None, SecurityPolicies.None, "opc.tcp://internal-host:4840/OtOpcUa") };
|
||||||
|
|
||||||
|
var best = EndpointSelector.SelectBest(endpoints, "opc.tcp://localhost", MessageSecurityMode.None);
|
||||||
|
|
||||||
|
var uri = new Uri(best.EndpointUrl);
|
||||||
|
uri.Host.ShouldBe("localhost");
|
||||||
|
uri.Port.ShouldBe(4840);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>An IPv6 host is rewritten with its brackets intact.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void SelectBest_RewritesIpv6Host()
|
||||||
|
{
|
||||||
|
var endpoints = new[] { Ep(MessageSecurityMode.None, SecurityPolicies.None, "opc.tcp://internal-host:4840/OtOpcUa") };
|
||||||
|
|
||||||
|
var best = EndpointSelector.SelectBest(endpoints, "opc.tcp://[::1]:4841", MessageSecurityMode.None);
|
||||||
|
|
||||||
|
var uri = new Uri(best.EndpointUrl);
|
||||||
|
uri.Host.ShouldBe("[::1]");
|
||||||
|
uri.Port.ShouldBe(4841);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+16
@@ -18,6 +18,11 @@ public class AggregateTypeMapperTests
|
|||||||
[InlineData(AggregateType.Start)]
|
[InlineData(AggregateType.Start)]
|
||||||
[InlineData(AggregateType.End)]
|
[InlineData(AggregateType.End)]
|
||||||
[InlineData(AggregateType.StandardDeviation)]
|
[InlineData(AggregateType.StandardDeviation)]
|
||||||
|
[InlineData(AggregateType.TimeAverage)]
|
||||||
|
[InlineData(AggregateType.TimeAverage2)]
|
||||||
|
[InlineData(AggregateType.Total)]
|
||||||
|
[InlineData(AggregateType.Total2)]
|
||||||
|
[InlineData(AggregateType.Interpolative)]
|
||||||
public void ToNodeId_ReturnsNonNullForAllValues(AggregateType aggregate)
|
public void ToNodeId_ReturnsNonNullForAllValues(AggregateType aggregate)
|
||||||
{
|
{
|
||||||
var nodeId = AggregateTypeMapper.ToNodeId(aggregate);
|
var nodeId = AggregateTypeMapper.ToNodeId(aggregate);
|
||||||
@@ -82,4 +87,15 @@ public class AggregateTypeMapperTests
|
|||||||
Should.Throw<ArgumentOutOfRangeException>(() =>
|
Should.Throw<ArgumentOutOfRangeException>(() =>
|
||||||
AggregateTypeMapper.ToNodeId((AggregateType)99));
|
AggregateTypeMapper.ToNodeId((AggregateType)99));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>#677.5: the time-weighted Part 13 aggregates map to their ObjectIds.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void ToNodeId_TimeWeightedAggregates_MapCorrectly()
|
||||||
|
{
|
||||||
|
AggregateTypeMapper.ToNodeId(AggregateType.TimeAverage).ShouldBe(ObjectIds.AggregateFunction_TimeAverage);
|
||||||
|
AggregateTypeMapper.ToNodeId(AggregateType.TimeAverage2).ShouldBe(ObjectIds.AggregateFunction_TimeAverage2);
|
||||||
|
AggregateTypeMapper.ToNodeId(AggregateType.Total).ShouldBe(ObjectIds.AggregateFunction_Total);
|
||||||
|
AggregateTypeMapper.ToNodeId(AggregateType.Total2).ShouldBe(ObjectIds.AggregateFunction_Total2);
|
||||||
|
AggregateTypeMapper.ToNodeId(AggregateType.Interpolative).ShouldBe(ObjectIds.AggregateFunction_Interpolative);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -136,6 +136,6 @@ public class ModelConstructionTests
|
|||||||
[Fact]
|
[Fact]
|
||||||
public void AggregateType_Enum_HasExpectedValues()
|
public void AggregateType_Enum_HasExpectedValues()
|
||||||
{
|
{
|
||||||
Enum.GetValues<AggregateType>().Length.ShouldBe(7);
|
Enum.GetValues<AggregateType>().Length.ShouldBe(12);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -141,7 +141,7 @@ public class HistoryViewModelTests
|
|||||||
public void AggregateTypes_ContainsNullForRaw()
|
public void AggregateTypes_ContainsNullForRaw()
|
||||||
{
|
{
|
||||||
_vm.AggregateTypes.ShouldContain((AggregateType?)null);
|
_vm.AggregateTypes.ShouldContain((AggregateType?)null);
|
||||||
_vm.AggregateTypes.Count.ShouldBe(8); // null + 7 enum values
|
_vm.AggregateTypes.Count.ShouldBe(13); // null + 12 enum values
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>Verifies that the Clear method resets the view model state.</summary>
|
/// <summary>Verifies that the Clear method resets the view model state.</summary>
|
||||||
|
|||||||
+25
-14
@@ -45,9 +45,10 @@ public sealed class IHistorianDataSourceContractTests
|
|||||||
[InlineData("ReadEventsAsync", typeof(Task<HistoricalEventsResult>))]
|
[InlineData("ReadEventsAsync", typeof(Task<HistoricalEventsResult>))]
|
||||||
public void ReadMethods_ReturnExpectedTaskShape(string methodName, Type expectedReturnType)
|
public void ReadMethods_ReturnExpectedTaskShape(string methodName, Type expectedReturnType)
|
||||||
{
|
{
|
||||||
var method = typeof(IHistorianDataSource).GetMethod(methodName);
|
// ReadEventsAsync is overloaded (the #675 direction-aware overload) — every overload must keep the shape.
|
||||||
method.ShouldNotBeNull();
|
var methods = Overloads(methodName);
|
||||||
method!.ReturnType.ShouldBe(expectedReturnType);
|
methods.ShouldNotBeEmpty();
|
||||||
|
foreach (var method in methods) method.ReturnType.ShouldBe(expectedReturnType);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>Verifies that GetHealthSnapshot is synchronous.</summary>
|
/// <summary>Verifies that GetHealthSnapshot is synchronous.</summary>
|
||||||
@@ -146,11 +147,14 @@ public sealed class IHistorianDataSourceContractTests
|
|||||||
[InlineData("ReadEventsAsync", "maxEvents", typeof(int))]
|
[InlineData("ReadEventsAsync", "maxEvents", typeof(int))]
|
||||||
public void HistoryRead_MaxParameter_TypePinned(string methodName, string parameterName, Type expectedType)
|
public void HistoryRead_MaxParameter_TypePinned(string methodName, string parameterName, Type expectedType)
|
||||||
{
|
{
|
||||||
var method = typeof(IHistorianDataSource).GetMethod(methodName);
|
var methods = Overloads(methodName);
|
||||||
method.ShouldNotBeNull();
|
methods.ShouldNotBeEmpty();
|
||||||
var parameter = method!.GetParameters().FirstOrDefault(p => p.Name == parameterName);
|
foreach (var method in methods)
|
||||||
parameter.ShouldNotBeNull($"Method {methodName} should expose a '{parameterName}' parameter.");
|
{
|
||||||
parameter!.ParameterType.ShouldBe(expectedType);
|
var parameter = method.GetParameters().FirstOrDefault(p => p.Name == parameterName);
|
||||||
|
parameter.ShouldNotBeNull($"Method {methodName} should expose a '{parameterName}' parameter.");
|
||||||
|
parameter!.ParameterType.ShouldBe(expectedType);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -163,13 +167,17 @@ public sealed class IHistorianDataSourceContractTests
|
|||||||
[Fact]
|
[Fact]
|
||||||
public void ReadEventsAsync_sentinel_and_continuation_contract_types_pinned()
|
public void ReadEventsAsync_sentinel_and_continuation_contract_types_pinned()
|
||||||
{
|
{
|
||||||
var method = typeof(IHistorianDataSource).GetMethod("ReadEventsAsync");
|
var methods = Overloads("ReadEventsAsync");
|
||||||
method.ShouldNotBeNull();
|
methods.Count.ShouldBe(2, "the five-argument read plus the #675 direction-aware overload");
|
||||||
|
methods.ShouldContain(m => m.GetParameters().Any(p => p.Name == "reverse" && p.ParameterType == typeof(bool)));
|
||||||
|
|
||||||
var maxEventsParam = method!.GetParameters().FirstOrDefault(p => p.Name == "maxEvents");
|
foreach (var method in methods)
|
||||||
maxEventsParam.ShouldNotBeNull("maxEvents parameter must exist for the sentinel contract");
|
{
|
||||||
maxEventsParam!.ParameterType.ShouldBe(typeof(int),
|
var maxEventsParam = method.GetParameters().FirstOrDefault(p => p.Name == "maxEvents");
|
||||||
"maxEvents is int (not uint) so callers can pass <=0 as a 'use backend default cap' sentinel");
|
maxEventsParam.ShouldNotBeNull("maxEvents parameter must exist for the sentinel contract");
|
||||||
|
maxEventsParam!.ParameterType.ShouldBe(typeof(int),
|
||||||
|
"maxEvents is int (not uint) so callers can pass <=0 as a 'use backend default cap' sentinel");
|
||||||
|
}
|
||||||
|
|
||||||
// HistoricalEventsResult must carry a nullable ContinuationPoint so implementations
|
// HistoricalEventsResult must carry a nullable ContinuationPoint so implementations
|
||||||
// can signal 'more results exist' (non-null) vs 'all results returned' (null).
|
// can signal 'more results exist' (non-null) vs 'all results returned' (null).
|
||||||
@@ -179,4 +187,7 @@ public sealed class IHistorianDataSourceContractTests
|
|||||||
continuationProp!.PropertyType.ShouldBe(typeof(byte[]),
|
continuationProp!.PropertyType.ShouldBe(typeof(byte[]),
|
||||||
"ContinuationPoint is byte[]? — null means no more pages, non-null means caller must page");
|
"ContinuationPoint is byte[]? — null means no more pages, non-null means caller must page");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static List<System.Reflection.MethodInfo> Overloads(string name) =>
|
||||||
|
typeof(IHistorianDataSource).GetMethods().Where(m => m.Name == name).ToList();
|
||||||
}
|
}
|
||||||
|
|||||||
+14
@@ -67,6 +67,20 @@ public sealed class NullHistorianDataSourceTests
|
|||||||
result.ContinuationPoint.ShouldBeNull();
|
result.ContinuationPoint.ShouldBeNull();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #675: the direction-aware overload's default — forward is the five-argument read, reverse is refused
|
||||||
|
/// (the server answers <c>BadHistoryOperationUnsupported</c>).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Direction_aware_ReadEventsAsync_forwards_forward_and_refuses_reverse()
|
||||||
|
{
|
||||||
|
IHistorianDataSource source = Source;
|
||||||
|
(await source.ReadEventsAsync(null, DateTime.UtcNow.AddHours(-1), DateTime.UtcNow, 0, reverse: false, Ct))
|
||||||
|
.Events.ShouldBeEmpty();
|
||||||
|
await Should.ThrowAsync<NotSupportedException>(() =>
|
||||||
|
source.ReadEventsAsync(null, DateTime.UtcNow, DateTime.UtcNow.AddHours(-1), 0, reverse: true, Ct));
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>GetHealthSnapshot reports a fully-disabled (zeroed, disconnected, no-node) shape.</summary>
|
/// <summary>GetHealthSnapshot reports a fully-disabled (zeroed, disconnected, no-node) shape.</summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
public void GetHealthSnapshot_is_disabled()
|
public void GetHealthSnapshot_is_disabled()
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The direction-aware event read (Gitea #675): forward is <c>[lo, hi)</c> ascending by <c>(EventTimeUtc, EventId)</c>;
|
||||||
|
/// reverse is the Part 11 reverse window <c>(lo, hi]</c> DESCENDING by the same key, so a capped reverse read returns the
|
||||||
|
/// LATEST events and flags truncation. Paging a reverse read the way the server does — resume at the last returned
|
||||||
|
/// time inclusive and drop the boundary ties already emitted — must visit every event exactly once, newest-first,
|
||||||
|
/// through a same-timestamp tie cluster.
|
||||||
|
/// </summary>
|
||||||
|
public abstract class EventReadConformance : ConformanceBase
|
||||||
|
{
|
||||||
|
private static readonly CancellationToken Ct = CancellationToken.None;
|
||||||
|
|
||||||
|
/// <summary>e0@0, e1@1, then the ties t0..t4@2, then e5@3 and e6@4 (all source "s"), plus o0@2 from another source.</summary>
|
||||||
|
private async Task<IHistorianDataSource> SeedAsync()
|
||||||
|
{
|
||||||
|
await B.SeedEventsAsync(
|
||||||
|
(At(0), "e0", "s"), (At(1), "e1", "s"),
|
||||||
|
(At(2), "t0", "s"), (At(2), "t1", "s"), (At(2), "t2", "s"), (At(2), "t3", "s"), (At(2), "t4", "s"),
|
||||||
|
(At(3), "e5", "s"), (At(4), "e6", "s"),
|
||||||
|
(At(2), "o0", "other"));
|
||||||
|
return B.CreateDataSource();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static readonly string[] NewestFirst = ["e6", "e5", "t4", "t3", "t2", "t1", "t0", "e1", "e0"];
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_read_is_newest_first_by_time_then_EventId_descending()
|
||||||
|
{
|
||||||
|
using var src = await SeedAsync();
|
||||||
|
var r = await src.ReadEventsAsync("s", At(-1), At(10), 0, reverse: true, Ct);
|
||||||
|
r.Events.Select(e => e.EventId).ShouldBe(NewestFirst);
|
||||||
|
r.ContinuationPoint.ShouldBeNull();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_window_is_end_exclusive_start_inclusive_whichever_order_the_bounds_come_in()
|
||||||
|
{
|
||||||
|
using var src = await SeedAsync();
|
||||||
|
// (At(1), At(3)]: e1 at the lower bound is excluded, e5 at the upper bound included.
|
||||||
|
(await src.ReadEventsAsync("s", At(3), At(1), 0, reverse: true, Ct)).Events.Select(e => e.EventId)
|
||||||
|
.ShouldBe(["e5", "t4", "t3", "t2", "t1", "t0"]);
|
||||||
|
(await src.ReadEventsAsync("s", At(1), At(3), 0, reverse: true, Ct)).Events.Select(e => e.EventId)
|
||||||
|
.ShouldBe(["e5", "t4", "t3", "t2", "t1", "t0"]);
|
||||||
|
// Forward is unchanged: [At(1), At(3)) ascending.
|
||||||
|
(await src.ReadEventsAsync("s", At(1), At(3), 0, reverse: false, Ct)).Events.Select(e => e.EventId)
|
||||||
|
.ShouldBe(["e1", "t0", "t1", "t2", "t3", "t4"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_capped_reverse_read_keeps_the_LATEST_events_and_flags_truncation()
|
||||||
|
{
|
||||||
|
using var src = await SeedAsync();
|
||||||
|
var r = await src.ReadEventsAsync("s", At(10), At(-1), 3, reverse: true, Ct);
|
||||||
|
r.Events.Select(e => e.EventId).ShouldBe(["e6", "e5", "t4"]);
|
||||||
|
r.ContinuationPoint.ShouldNotBeNull();
|
||||||
|
// An exact fit is complete.
|
||||||
|
(await src.ReadEventsAsync("s", At(10), At(-1), 9, reverse: true, Ct)).ContinuationPoint.ShouldBeNull();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Paging_a_reverse_read_through_a_tie_cluster_visits_every_event_once_newest_first()
|
||||||
|
{
|
||||||
|
using var src = await SeedAsync();
|
||||||
|
foreach (var cap in new[] { 1, 2, 3, 4 })
|
||||||
|
{
|
||||||
|
var all = new List<string>();
|
||||||
|
var lo = At(-1);
|
||||||
|
var from = At(10);
|
||||||
|
var skip = 0;
|
||||||
|
for (var page = 0; ; page++)
|
||||||
|
{
|
||||||
|
page.ShouldBeLessThan(40, "paging must terminate");
|
||||||
|
var r = await src.ReadEventsAsync("s", from, lo, cap + skip, reverse: true, Ct);
|
||||||
|
var events = r.Events.ToList();
|
||||||
|
var drop = 0;
|
||||||
|
while (drop < skip && drop < events.Count && events[drop].EventTimeUtc == from) drop++;
|
||||||
|
events = events.Skip(drop).ToList();
|
||||||
|
// The server asks for cap + skip rows (it re-reads the boundary ties it then drops), so a page is never
|
||||||
|
// emptied by the trim while more remains.
|
||||||
|
if (r.ContinuationPoint is not null) events.Count.ShouldBe(cap);
|
||||||
|
|
||||||
|
all.AddRange(events.Select(e => e.EventId));
|
||||||
|
if (r.ContinuationPoint is null) break;
|
||||||
|
var boundary = events[^1].EventTimeUtc;
|
||||||
|
var ties = events.AsEnumerable().Reverse().TakeWhile(e => e.EventTimeUtc == boundary).Count();
|
||||||
|
skip = boundary == from ? skip + ties : ties;
|
||||||
|
from = boundary;
|
||||||
|
}
|
||||||
|
|
||||||
|
all.SequenceEqual(NewestFirst).ShouldBeTrue($"cap {cap}: {string.Join(",", all)}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_read_with_a_null_source_spans_every_source()
|
||||||
|
{
|
||||||
|
using var src = await SeedAsync();
|
||||||
|
var r = await src.ReadEventsAsync(null, At(2.5), At(1.5), 0, reverse: true, Ct);
|
||||||
|
r.Events.Select(e => e.EventId).ShouldBe(["t4", "t3", "t2", "t1", "t0", "o0"]);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -41,6 +41,13 @@ public interface IConformanceBackend : IAsyncDisposable
|
|||||||
/// <param name="samples">The rows to store.</param>
|
/// <param name="samples">The rows to store.</param>
|
||||||
Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples);
|
Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Writes historical events directly in the backend's event table (never through the alarm writer): event id,
|
||||||
|
/// source and time as given (rounded to the storage resolution), severity 500, received time = event time.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="events">The events to store.</param>
|
||||||
|
Task SeedEventsAsync(params (DateTime T, string Id, string Source)[] events);
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Live (non-deleted) rows for a tag, ascending by time, read directly from storage. The status is returned
|
/// Live (non-deleted) rows for a tag, ascending by time, read directly from storage. The status is returned
|
||||||
/// VERBATIM as stored — there is no read-side clamp (an Empty row stored non-Bad reads back non-Bad here).
|
/// VERBATIM as stored — there is no read-side clamp (an Empty row stored non-Bad reads back non-Bad here).
|
||||||
|
|||||||
+15
-6
@@ -71,7 +71,7 @@ public sealed class WireBackendTests
|
|||||||
await driver.InitializeAsync("{}", ct);
|
await driver.InitializeAsync("{}", ct);
|
||||||
|
|
||||||
await WaitFor(() =>
|
await WaitFor(() =>
|
||||||
driver.GetDeviceState(DeviceHost) is { FixedTreeCache: not null }, TimeSpan.FromSeconds(5));
|
driver.GetDeviceState(DeviceHost) is { FixedTreeCache: not null }, WaitTimeout);
|
||||||
|
|
||||||
var state = driver.GetDeviceState(DeviceHost);
|
var state = driver.GetDeviceState(DeviceHost);
|
||||||
state.ShouldNotBeNull();
|
state.ShouldNotBeNull();
|
||||||
@@ -82,7 +82,7 @@ public sealed class WireBackendTests
|
|||||||
|
|
||||||
await WaitFor(() =>
|
await WaitFor(() =>
|
||||||
state.LastFixedSnapshots.ContainsKey($"{DeviceHost}/Axes/X/AbsolutePosition"),
|
state.LastFixedSnapshots.ContainsKey($"{DeviceHost}/Axes/X/AbsolutePosition"),
|
||||||
TimeSpan.FromSeconds(3));
|
WaitTimeout);
|
||||||
state.LastFixedSnapshots[$"{DeviceHost}/Axes/X/AbsolutePosition"].ShouldBe(123456);
|
state.LastFixedSnapshots[$"{DeviceHost}/Axes/X/AbsolutePosition"].ShouldBe(123456);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -136,7 +136,7 @@ public sealed class WireBackendTests
|
|||||||
|
|
||||||
await WaitFor(() =>
|
await WaitFor(() =>
|
||||||
driver.GetDeviceState(DeviceHost) is { LastProgramInfo: not null },
|
driver.GetDeviceState(DeviceHost) is { LastProgramInfo: not null },
|
||||||
TimeSpan.FromSeconds(5));
|
WaitTimeout);
|
||||||
|
|
||||||
var snapshots = await driver.ReadAsync(
|
var snapshots = await driver.ReadAsync(
|
||||||
[$"{DeviceHost}/Program/Name",
|
[$"{DeviceHost}/Program/Name",
|
||||||
@@ -194,7 +194,7 @@ public sealed class WireBackendTests
|
|||||||
{
|
{
|
||||||
var state = driver.GetDeviceState(DeviceHost);
|
var state = driver.GetDeviceState(DeviceHost);
|
||||||
return state is not null && state.LastTimers.Count == 4;
|
return state is not null && state.LastTimers.Count == 4;
|
||||||
}, TimeSpan.FromSeconds(5));
|
}, WaitTimeout);
|
||||||
|
|
||||||
var snapshots = await driver.ReadAsync(
|
var snapshots = await driver.ReadAsync(
|
||||||
[$"{DeviceHost}/Timers/PowerOnSeconds",
|
[$"{DeviceHost}/Timers/PowerOnSeconds",
|
||||||
@@ -265,8 +265,11 @@ public sealed class WireBackendTests
|
|||||||
{
|
{
|
||||||
var state = driver.GetDeviceState(DeviceHost);
|
var state = driver.GetDeviceState(DeviceHost);
|
||||||
return state?.FixedTreeCache is { Capabilities.SpindleLoad: true, Capabilities.SpindleMotorSpeed: true }
|
return state?.FixedTreeCache is { Capabilities.SpindleLoad: true, Capabilities.SpindleMotorSpeed: true }
|
||||||
&& state.LastSpindleLoads.Count >= 2;
|
&& state.LastSpindleLoads.Count >= 2
|
||||||
}, TimeSpan.FromSeconds(5));
|
// Motor speeds are cached after a SECOND wire round-trip (FocasDriver spindle poll), so the loads
|
||||||
|
// arriving first does not mean the speeds have — waiting on loads alone raced under load (#678.3).
|
||||||
|
&& state.LastSpindleMotorSpeeds.Count >= 2;
|
||||||
|
}, WaitTimeout);
|
||||||
|
|
||||||
var snapshots = await driver.ReadAsync(
|
var snapshots = await driver.ReadAsync(
|
||||||
[$"{DeviceHost}/Spindle/S1/Load",
|
[$"{DeviceHost}/Spindle/S1/Load",
|
||||||
@@ -282,6 +285,10 @@ public sealed class WireBackendTests
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>Generous: the sim fixture answers fast, but a loaded CI host (e.g. a concurrent docker build) does not.</summary>
|
||||||
|
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(15);
|
||||||
|
|
||||||
|
/// <summary>Polls <paramref name="pred"/>; FAILS the test with a clear message on timeout rather than returning silently.</summary>
|
||||||
private static async Task WaitFor(Func<bool> pred, TimeSpan timeout)
|
private static async Task WaitFor(Func<bool> pred, TimeSpan timeout)
|
||||||
{
|
{
|
||||||
var deadline = DateTime.UtcNow + timeout;
|
var deadline = DateTime.UtcNow + timeout;
|
||||||
@@ -290,5 +297,7 @@ public sealed class WireBackendTests
|
|||||||
if (pred()) return;
|
if (pred()) return;
|
||||||
await Task.Delay(50);
|
await Task.Delay(50);
|
||||||
}
|
}
|
||||||
|
if (pred()) return;
|
||||||
|
Assert.Fail($"Condition not met within {timeout.TotalSeconds:0.#}s waiting on the FOCAS wire backend.");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+123
@@ -0,0 +1,123 @@
|
|||||||
|
using System.Net;
|
||||||
|
using Google.Protobuf.WellKnownTypes;
|
||||||
|
using Grpc.Core;
|
||||||
|
using Microsoft.AspNetCore.Builder;
|
||||||
|
using Microsoft.AspNetCore.Hosting;
|
||||||
|
using Microsoft.AspNetCore.Server.Kestrel.Core;
|
||||||
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.HistorianGateway.Contracts.Grpc;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.Tests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #675: the Gateway backend's event-read bounds and direction, offline.
|
||||||
|
/// <list type="bullet">
|
||||||
|
/// <item>A reverse (newest-first) read is refused before anything is sent — the gateway streams ascending only.</item>
|
||||||
|
/// <item>The open upper bound a start-only forward read uses (<c>DateTime.MaxValue</c>, UTC kind) survives the
|
||||||
|
/// PUBLISHED client package's proto conversion — captured on the wire by an in-process gRPC server — and the
|
||||||
|
/// Windows FILETIME encoding the gateway converts it to. The live AVEVA leg (what the historian does with a
|
||||||
|
/// year-9999 bound) stays open: it needs the VPN.</item>
|
||||||
|
/// </list>
|
||||||
|
/// </summary>
|
||||||
|
public sealed class GatewayEventBoundsTests
|
||||||
|
{
|
||||||
|
private static readonly DateTime UtcMax = new(DateTime.MaxValue.Ticks, DateTimeKind.Utc);
|
||||||
|
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_reverse_events_read_is_refused_and_sends_nothing()
|
||||||
|
{
|
||||||
|
var fake = new FakeHistorianGatewayClient();
|
||||||
|
IHistorianDataSource ds = new GatewayHistorianDataSource(fake, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||||
|
|
||||||
|
await Should.ThrowAsync<NotSupportedException>(() =>
|
||||||
|
ds.ReadEventsAsync("Pump1", DateTime.UtcNow, DateTime.UtcNow.AddHours(-1), 10, reverse: true, Ct));
|
||||||
|
fake.ReadEventsCallCount.ShouldBe(0);
|
||||||
|
|
||||||
|
// Forward through the same overload is the five-argument read, unchanged.
|
||||||
|
await ds.ReadEventsAsync("Pump1", DateTime.UtcNow.AddHours(-1), DateTime.UtcNow, 10, reverse: false, Ct);
|
||||||
|
fake.ReadEventsCallCount.ShouldBe(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void UtcMax_round_trips_through_the_proto_Timestamp_and_FILETIME_encodings()
|
||||||
|
{
|
||||||
|
var ts = Timestamp.FromDateTime(UtcMax);
|
||||||
|
ts.ToDateTime().ShouldBe(UtcMax);
|
||||||
|
ts.ToDateTime().Kind.ShouldBe(DateTimeKind.Utc);
|
||||||
|
|
||||||
|
// The gateway hands its bounds to the historian as FILETIME (100 ns since 1601) — in range up to year 30828.
|
||||||
|
var fileTime = ts.ToDateTime().ToFileTimeUtc();
|
||||||
|
DateTime.FromFileTimeUtc(fileTime).ShouldBe(UtcMax);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_start_only_bound_of_UtcMax_reaches_the_wire_intact_through_the_published_client()
|
||||||
|
{
|
||||||
|
var capture = new CapturingReadService();
|
||||||
|
await using var server = await StartServerAsync(capture);
|
||||||
|
using var adapter = HistorianGatewayClientAdapter.Create(
|
||||||
|
new ServerHistorianOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Endpoint = $"http://127.0.0.1:{server.Port}",
|
||||||
|
ApiKey = "histgw_test_key",
|
||||||
|
UseTls = false,
|
||||||
|
},
|
||||||
|
NullLoggerFactory.Instance);
|
||||||
|
IHistorianDataSource ds = new GatewayHistorianDataSource(adapter, NullLogger<GatewayHistorianDataSource>.Instance);
|
||||||
|
|
||||||
|
var start = new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
|
||||||
|
var r = await ds.ReadEventsAsync("Pump1", start, UtcMax, 5, Ct);
|
||||||
|
|
||||||
|
r.Events.ShouldBeEmpty();
|
||||||
|
capture.Request.ShouldNotBeNull();
|
||||||
|
capture.Request!.StartTime.ToDateTime().ShouldBe(start);
|
||||||
|
capture.Request.EndTime.ToDateTime().ShouldBe(UtcMax);
|
||||||
|
DateTime.FromFileTimeUtc(capture.Request.EndTime.ToDateTime().ToFileTimeUtc()).ShouldBe(UtcMax);
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class CapturingReadService : HistorianRead.HistorianReadBase
|
||||||
|
{
|
||||||
|
public ReadEventsRequest? Request { get; private set; }
|
||||||
|
|
||||||
|
public override Task ReadEvents(
|
||||||
|
ReadEventsRequest request, IServerStreamWriter<HistorianEvent> responseStream, ServerCallContext context)
|
||||||
|
{
|
||||||
|
Request = request;
|
||||||
|
return Task.CompletedTask;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class Server(WebApplication app, int port) : IAsyncDisposable
|
||||||
|
{
|
||||||
|
public int Port { get; } = port;
|
||||||
|
|
||||||
|
public async ValueTask DisposeAsync()
|
||||||
|
{
|
||||||
|
await app.StopAsync();
|
||||||
|
await app.DisposeAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static async Task<Server> StartServerAsync(CapturingReadService service)
|
||||||
|
{
|
||||||
|
var builder = WebApplication.CreateSlimBuilder();
|
||||||
|
builder.Logging.ClearProviders();
|
||||||
|
builder.WebHost.ConfigureKestrel(k =>
|
||||||
|
k.Listen(IPAddress.Loopback, 0, o => o.Protocols = HttpProtocols.Http2));
|
||||||
|
builder.Services.AddGrpc();
|
||||||
|
builder.Services.AddSingleton(service);
|
||||||
|
var app = builder.Build();
|
||||||
|
app.MapGrpcService<CapturingReadService>();
|
||||||
|
await app.StartAsync();
|
||||||
|
var port = new Uri(app.Urls.First()).Port;
|
||||||
|
return new Server(app, port);
|
||||||
|
}
|
||||||
|
}
|
||||||
+6
@@ -15,12 +15,18 @@
|
|||||||
<PackageReference Include="Microsoft.NET.Test.Sdk"/>
|
<PackageReference Include="Microsoft.NET.Test.Sdk"/>
|
||||||
<!-- ServiceCollection + BuildServiceProvider for the CreateDataSource factory test. -->
|
<!-- ServiceCollection + BuildServiceProvider for the CreateDataSource factory test. -->
|
||||||
<PackageReference Include="Microsoft.Extensions.DependencyInjection"/>
|
<PackageReference Include="Microsoft.Extensions.DependencyInjection"/>
|
||||||
|
<!-- In-process gRPC server capturing the published client's ReadEvents request (#675 bounds pin). -->
|
||||||
|
<PackageReference Include="Grpc.AspNetCore"/>
|
||||||
<PackageReference Include="xunit.runner.visualstudio">
|
<PackageReference Include="xunit.runner.visualstudio">
|
||||||
<PrivateAssets>all</PrivateAssets>
|
<PrivateAssets>all</PrivateAssets>
|
||||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||||
</PackageReference>
|
</PackageReference>
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
|
<ItemGroup>
|
||||||
|
<FrameworkReference Include="Microsoft.AspNetCore.App"/>
|
||||||
|
</ItemGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<ProjectReference Include="..\..\..\src\Drivers\ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway\ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.csproj"/>
|
<ProjectReference Include="..\..\..\src\Drivers\ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway\ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway.csproj"/>
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|||||||
+7
@@ -60,3 +60,10 @@ public sealed class QuestDbModifiedValueConformance(QuestDbFixture fx) : Modifie
|
|||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
|
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Collection(QuestDbCollection.Name)]
|
||||||
|
public sealed class QuestDbEventReadConformance(QuestDbFixture fx) : EventReadConformance
|
||||||
|
{
|
||||||
|
/// <inheritdoc/>
|
||||||
|
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
|
||||||
|
}
|
||||||
|
|||||||
+21
@@ -143,6 +143,27 @@ internal sealed class QuestDbConformanceBackend : IConformanceBackend
|
|||||||
await Wal.WaitAsync(Conn.SampleTable, CancellationToken.None);
|
await Wal.WaitAsync(Conn.SampleTable, CancellationToken.None);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public async Task SeedEventsAsync(params (DateTime T, string Id, string Source)[] events)
|
||||||
|
{
|
||||||
|
await using (var c = await Conn.OpenAsync(CancellationToken.None))
|
||||||
|
{
|
||||||
|
foreach (var (t, id, source) in events)
|
||||||
|
{
|
||||||
|
await using var cmd = Conn.CreateCommand(c,
|
||||||
|
$"INSERT INTO {Conn.EventTable} (ts, event_id, source, severity, message, received_utc) VALUES ($1, $2, $3, $4, $5, $6)");
|
||||||
|
QuestDbConnection.BindTimestamp(cmd, t);
|
||||||
|
QuestDbConnection.BindText(cmd, id);
|
||||||
|
QuestDbConnection.BindText(cmd, source);
|
||||||
|
cmd.Parameters.Add(new NpgsqlParameter { NpgsqlDbType = NpgsqlDbType.Integer, Value = 500 });
|
||||||
|
QuestDbConnection.BindText(cmd, null);
|
||||||
|
QuestDbConnection.BindTimestamp(cmd, t);
|
||||||
|
await cmd.ExecuteNonQueryAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await Wal.WaitAsync(Conn.EventTable, CancellationToken.None);
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>Runs a statement against this table set (test-only escape hatch for hand-made rows), then waits for WAL visibility of the sample table.</summary>
|
/// <summary>Runs a statement against this table set (test-only escape hatch for hand-made rows), then waits for WAL visibility of the sample table.</summary>
|
||||||
/// <param name="sql">The statement; parameters are bound positionally by <paramref name="bind"/>.</param>
|
/// <param name="sql">The statement; parameters are bound positionally by <paramref name="bind"/>.</param>
|
||||||
/// <param name="bind">Binds the parameters.</param>
|
/// <param name="bind">Binds the parameters.</param>
|
||||||
|
|||||||
+80
@@ -0,0 +1,80 @@
|
|||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674.1 — pins the QuestDB TTL behaviour the historian's retention relies on: a table TTL is judged against
|
||||||
|
/// the wall clock (<c>min(max timestamp, now)</c>), so one far-future row — a 2066 typo that reached the backend
|
||||||
|
/// without going through the node manager's <c>HistoryUpdateMaxFutureSkew</c> bound — cannot age out every older
|
||||||
|
/// partition, including the Part 11 modification trail (the record of who changed history).
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// <b>Positive control.</b> A partition genuinely older than the TTL (40 days, against a 30-day TTL) is kept until
|
||||||
|
/// the far-future commit and evicted BY it — proving TTL housekeeping ran on exactly the commit that could have
|
||||||
|
/// dropped the in-TTL partitions. Without it the "nothing evicted" assertion would also pass against a table
|
||||||
|
/// whose TTL never ran.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// ⚠️ Observed on QuestDB 10.0.1 (2026-10-11): the in-TTL partitions also survived with
|
||||||
|
/// <c>cairo.ttl.use.wall.clock=false</c> (verified effective via <c>SHOW PARAMETERS</c>), so this test does not
|
||||||
|
/// isolate that flag — it pins the outcome. Keep the flag at its default (<c>true</c>) regardless; the server-side
|
||||||
|
/// future-skew bound is the primary guard.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
[Collection(QuestDbCollection.Name)]
|
||||||
|
public sealed class TtlWallClockTests(QuestDbFixture fx)
|
||||||
|
{
|
||||||
|
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||||
|
private static readonly HistoryUpdateContext Who = new("ttl-test", null);
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_far_future_row_evicts_only_partitions_older_than_the_ttl_by_the_wall_clock()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var o = fx.Options(fx.UniquePrefix(), retentionDays: 30);
|
||||||
|
var c = new QuestDbConnection(o);
|
||||||
|
var wal = new QuestDbWalVisibility(c, o.VisibilityTimeout);
|
||||||
|
var tags = new QuestDbTagCatalog(c, new QuestDbSchemaBootstrapper(c, o), wal);
|
||||||
|
var updater = new QuestDbHistorianDataUpdater(c, tags, wal);
|
||||||
|
|
||||||
|
// Whole seconds (µs-aligned), one row per DAY partition. With the newest row 10 days old, the 40-day partition is
|
||||||
|
// still inside the TTL measured from the table's max timestamp, so this commit evicts nothing.
|
||||||
|
var now = DateTime.UtcNow;
|
||||||
|
var today = new DateTime(now.Year, now.Month, now.Day, now.Hour, now.Minute, now.Second, DateTimeKind.Utc);
|
||||||
|
DateTime[] older = [today.AddDays(-40), today.AddDays(-20), today.AddDays(-10)];
|
||||||
|
(await updater.UpdateDataAsync("ttl.tag", HistoryUpdateMode.Insert, older.Select((t, i) => V(t, i)).ToList(), Who, Ct))
|
||||||
|
.ShouldAllBe(code => code == HistorianStatusCodes.GoodEntryInserted);
|
||||||
|
(await DaysAsync(c, c.SampleTable)).ShouldBe(older.Select(t => t.Date));
|
||||||
|
(await DaysAsync(c, c.SampleModifiedTable)).ShouldBe(older.Select(t => t.Date));
|
||||||
|
|
||||||
|
// The far-future row, then one more ordinary commit (TTL housekeeping runs on commits).
|
||||||
|
(await updater.UpdateDataAsync("ttl.tag", HistoryUpdateMode.Insert,
|
||||||
|
[V(new DateTime(2066, 1, 1, 0, 0, 0, DateTimeKind.Utc), 9)], Who, Ct)).ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||||
|
(await updater.UpdateDataAsync("ttl.tag", HistoryUpdateMode.Insert, [V(today, 10)], Who, Ct))
|
||||||
|
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||||
|
(await wal.WaitAsync(c.SampleTable, Ct)).ShouldBeTrue();
|
||||||
|
(await wal.WaitAsync(c.SampleModifiedTable, Ct)).ShouldBeTrue();
|
||||||
|
|
||||||
|
// Only the genuinely expired 40-day partition is gone; the 20- and 10-day partitions survive in both tables.
|
||||||
|
DateTime[] expected = [today.AddDays(-20).Date, today.AddDays(-10).Date, today.Date, new(2066, 1, 1)];
|
||||||
|
(await DaysAsync(c, c.SampleTable)).ShouldBe(expected);
|
||||||
|
(await DaysAsync(c, c.SampleModifiedTable)).ShouldBe(expected);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static HistorizationValue V(DateTime t, double v) => new(t, HistorianValue.FromDouble(v), HistorianStatusCodes.Good);
|
||||||
|
|
||||||
|
/// <summary>The distinct UTC days holding rows, ascending.</summary>
|
||||||
|
private static async Task<List<DateTime>> DaysAsync(QuestDbConnection c, string table)
|
||||||
|
{
|
||||||
|
await using var conn = await c.OpenAsync(Ct);
|
||||||
|
await using var cmd = c.CreateCommand(conn, $"SELECT ts FROM {table} ORDER BY ts");
|
||||||
|
await using var r = await cmd.ExecuteReaderAsync(Ct);
|
||||||
|
var days = new SortedSet<DateTime>();
|
||||||
|
while (await r.ReadAsync(Ct)) days.Add(QuestDbConnection.ReadUtc(r, 0).Date);
|
||||||
|
return days.ToList();
|
||||||
|
}
|
||||||
|
}
|
||||||
+5
@@ -14,6 +14,11 @@ public sealed class StatusCodeOracleTests
|
|||||||
((uint)Opc.Ua.AggregateBits.Calculated).ShouldBe(HistorianInfoBits.HistorianCalculated);
|
((uint)Opc.Ua.AggregateBits.Calculated).ShouldBe(HistorianInfoBits.HistorianCalculated);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>#674: the v1-schema refusal code is the SDK's BadHistoryOperationUnsupported.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void History_operation_unsupported_matches_the_SDK() =>
|
||||||
|
HistorianStatusCodes.BadHistoryOperationUnsupported.ShouldBe(Opc.Ua.StatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public void Interpolated_bits_match_the_SDK()
|
public void Interpolated_bits_match_the_SDK()
|
||||||
{
|
{
|
||||||
|
|||||||
+3
-3
@@ -54,7 +54,7 @@ public sealed class ModifiedHistoryAtomicityTests(SqlHistorianFixture fx)
|
|||||||
public async Task A_replace_whose_trail_insert_fails_changes_nothing()
|
public async Task A_replace_whose_trail_insert_fails_changes_nothing()
|
||||||
{
|
{
|
||||||
var db = await ReadTestDb.CreateAsync(fx);
|
var db = await ReadTestDb.CreateAsync(fx);
|
||||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
var u = new SqlServerHistorianDataUpdater(db.Conn, db.Boot, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], Who, CancellationToken.None))
|
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], Who, CancellationToken.None))
|
||||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||||
(await CountTrailAsync(db)).ShouldBe(1);
|
(await CountTrailAsync(db)).ShouldBe(1);
|
||||||
@@ -78,7 +78,7 @@ public sealed class ModifiedHistoryAtomicityTests(SqlHistorianFixture fx)
|
|||||||
public async Task Deletes_whose_trail_insert_fails_delete_nothing()
|
public async Task Deletes_whose_trail_insert_fails_delete_nothing()
|
||||||
{
|
{
|
||||||
var db = await ReadTestDb.CreateAsync(fx);
|
var db = await ReadTestDb.CreateAsync(fx);
|
||||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
var u = new SqlServerHistorianDataUpdater(db.Conn, db.Boot, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1), V(T2, 2)], Who, CancellationToken.None);
|
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1), V(T2, 2)], Who, CancellationToken.None);
|
||||||
|
|
||||||
await BreakTheTrailAsync(db);
|
await BreakTheTrailAsync(db);
|
||||||
@@ -99,7 +99,7 @@ public sealed class ModifiedHistoryAtomicityTests(SqlHistorianFixture fx)
|
|||||||
public async Task An_over_long_principal_is_truncated_never_refused()
|
public async Task An_over_long_principal_is_truncated_never_refused()
|
||||||
{
|
{
|
||||||
var db = await ReadTestDb.CreateAsync(fx);
|
var db = await ReadTestDb.CreateAsync(fx);
|
||||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
var u = new SqlServerHistorianDataUpdater(db.Conn, db.Boot, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||||
var who = new HistoryUpdateContext(new string('p', 500), new string('k', 500));
|
var who = new HistoryUpdateContext(new string('p', 500), new string('k', 500));
|
||||||
|
|
||||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], who, CancellationToken.None))
|
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], who, CancellationToken.None))
|
||||||
|
|||||||
+91
@@ -178,6 +178,97 @@ public sealed class RetentionTests(SqlHistorianFixture fx)
|
|||||||
(await LockIsFreeAsync(s)).ShouldBeTrue();
|
(await LockIsFreeAsync(s)).ShouldBeTrue();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Cancelling_mid_command_while_the_delete_is_blocked_still_releases_the_applock()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var (c, b, _, s) = Make();
|
||||||
|
await SeedAsync(b, s);
|
||||||
|
|
||||||
|
// A second session holds an exclusive table lock, so the purger's first DELETE blocks server-side.
|
||||||
|
await using var holder = new SqlConnection(fx.ConnectionString);
|
||||||
|
await holder.OpenAsync();
|
||||||
|
await using var tx = (SqlTransaction)await holder.BeginTransactionAsync();
|
||||||
|
await using (var cmd = holder.CreateCommand())
|
||||||
|
{
|
||||||
|
cmd.Transaction = tx;
|
||||||
|
cmd.CommandText = $"SELECT COUNT(*) FROM [{s}].Sample WITH (TABLOCKX, HOLDLOCK)";
|
||||||
|
await cmd.ExecuteScalarAsync();
|
||||||
|
}
|
||||||
|
short holderSpid;
|
||||||
|
await using (var cmd = holder.CreateCommand())
|
||||||
|
{
|
||||||
|
cmd.Transaction = tx;
|
||||||
|
cmd.CommandText = "SELECT CAST(@@SPID AS smallint)";
|
||||||
|
holderSpid = (short)(await cmd.ExecuteScalarAsync())!;
|
||||||
|
}
|
||||||
|
|
||||||
|
using var cts = new CancellationTokenSource();
|
||||||
|
var purger = new SqlServerHistorianRetention(c, b, TimeSpan.FromHours(1), 30, 4000, time: new FixedTime(Now));
|
||||||
|
var purge = purger.PurgeAsync(cts.Token);
|
||||||
|
|
||||||
|
// Cancel only once the server reports a DELETE actually blocked behind the holder — a fixed delay could
|
||||||
|
// cancel during bootstrap/applock acquisition on a slow host and never exercise the mid-command path.
|
||||||
|
var deadline = DateTime.UtcNow.AddSeconds(15);
|
||||||
|
while (await CountAsync(
|
||||||
|
$"SELECT COUNT(*) FROM sys.dm_exec_requests WHERE blocking_session_id = {holderSpid} AND command = 'DELETE'") == 0)
|
||||||
|
{
|
||||||
|
purge.IsCompleted.ShouldBeFalse("the purge finished without ever blocking on the holder's table lock");
|
||||||
|
DateTime.UtcNow.ShouldBeLessThan(deadline, "the purge's DELETE never blocked behind the holder");
|
||||||
|
await Task.Delay(50);
|
||||||
|
}
|
||||||
|
cts.Cancel();
|
||||||
|
|
||||||
|
// SqlClient surfaces the attention as OperationCanceledException or as a SqlException.
|
||||||
|
var ex = await Should.ThrowAsync<Exception>(() => purge);
|
||||||
|
(ex is OperationCanceledException or SqlException).ShouldBeTrue($"unexpected {ex.GetType().Name}: {ex.Message}");
|
||||||
|
|
||||||
|
await tx.RollbackAsync();
|
||||||
|
(await LockIsFreeAsync(s)).ShouldBeTrue();
|
||||||
|
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Sample")).ShouldBe(21);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_failing_release_falls_back_to_clearing_the_pool_so_the_applock_is_freed()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var (c, b, _, s) = Make();
|
||||||
|
await SeedAsync(b, s);
|
||||||
|
var purger = new SqlServerHistorianRetention(c, b, TimeSpan.FromHours(1), 30, 4000, time: new FixedTime(Now))
|
||||||
|
{
|
||||||
|
ReleaseSql = "EXEC sp_this_procedure_does_not_exist;",
|
||||||
|
};
|
||||||
|
|
||||||
|
await purger.PurgeAsync(CancellationToken.None);
|
||||||
|
|
||||||
|
(await LockIsFreeAsync(s)).ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Falsification_control_a_failing_release_with_a_no_op_ClearPool_leaves_the_applock_held()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var (c, b, _, s) = Make();
|
||||||
|
await SeedAsync(b, s);
|
||||||
|
var purger = new SqlServerHistorianRetention(c, b, TimeSpan.FromHours(1), 30, 4000, time: new FixedTime(Now))
|
||||||
|
{
|
||||||
|
ReleaseSql = "EXEC sp_this_procedure_does_not_exist;",
|
||||||
|
ClearPool = _ => { },
|
||||||
|
};
|
||||||
|
|
||||||
|
await purger.PurgeAsync(CancellationToken.None);
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// The pooled session still owns the Session-scoped lock: proves the fallback test above is not vacuous.
|
||||||
|
(await LockIsFreeAsync(s)).ShouldBeFalse();
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
SqlConnection.ClearAllPools(); // drop the leaked session so the lock does not outlive the test
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task A_throwing_purge_releases_the_applock()
|
public async Task A_throwing_purge_releases_the_applock()
|
||||||
{
|
{
|
||||||
|
|||||||
+14
-9
@@ -32,7 +32,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
|||||||
var s = SqlHistorianFixture.UniqueSchema();
|
var s = SqlHistorianFixture.UniqueSchema();
|
||||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None);
|
await Boot(Opts(s)).EnsureAsync(CancellationToken.None);
|
||||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None); // fresh instance => re-runs the guarded path
|
await Boot(Opts(s)).EnsureAsync(CancellationToken.None); // fresh instance => re-runs the guarded path
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.ModifiedTrailVersion);
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM sys.indexes WHERE name='CX_Event_Time' AND object_id=OBJECT_ID('[{s}].Event')")).ShouldBe(1);
|
(await ScalarAsync($"SELECT COUNT(*) FROM sys.indexes WHERE name='CX_Event_Time' AND object_id=OBJECT_ID('[{s}].Event')")).ShouldBe(1);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -51,13 +51,14 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
|||||||
cmd.CommandText = b;
|
cmd.CommandText = b;
|
||||||
await cmd.ExecuteNonQueryAsync();
|
await cmd.ExecuteNonQueryAsync();
|
||||||
}
|
}
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.ModifiedTrailVersion);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// #668 upgrade path: a schema left at version 1 (only <c>001_initial.sql</c> applied, as every pre-#668 install is)
|
/// #668 upgrade path: a schema left at version 1 (only <c>001_initial.sql</c> applied, as every pre-#668 install is)
|
||||||
/// is upgraded in place by an auto-creating bootstrap — <c>SampleModified</c> appears, both version rows exist, and
|
/// is upgraded in place by an auto-creating bootstrap — <c>SampleModified</c> appears, both version rows exist, and
|
||||||
/// existing samples are untouched. With <c>AutoCreateSchema=false</c> the same v1 schema is refused.
|
/// existing samples are untouched. With <c>AutoCreateSchema=false</c> the same v1 schema is accepted with the
|
||||||
|
/// modification trail switched off and left at v1 (#674).
|
||||||
/// </summary>
|
/// </summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task A_version_1_schema_is_upgraded_to_version_2_in_place()
|
public async Task A_version_1_schema_is_upgraded_to_version_2_in_place()
|
||||||
@@ -81,11 +82,15 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
|||||||
}
|
}
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM sys.tables WHERE name = 'SampleModified' AND schema_id = SCHEMA_ID('{s}')")).ShouldBe(0);
|
(await ScalarAsync($"SELECT COUNT(*) FROM sys.tables WHERE name = 'SampleModified' AND schema_id = SCHEMA_ID('{s}')")).ShouldBe(0);
|
||||||
|
|
||||||
var refused = await Should.ThrowAsync<HistorianSchemaMissingException>(
|
// #674: verify-only accepts v1 — with the modification trail switched off — and does not upgrade it.
|
||||||
() => Boot(Opts(s, auto: false)).EnsureAsync(CancellationToken.None));
|
var verifyOnly = Boot(Opts(s, auto: false));
|
||||||
refused.Message.ShouldContain("missing version 2 (found 1)");
|
await verifyOnly.EnsureAsync(CancellationToken.None);
|
||||||
|
verifyOnly.SupportsModifiedTrail.ShouldBeFalse();
|
||||||
|
(await ScalarAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(1);
|
||||||
|
|
||||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None);
|
var upgrading = Boot(Opts(s));
|
||||||
|
await upgrading.EnsureAsync(CancellationToken.None);
|
||||||
|
upgrading.SupportsModifiedTrail.ShouldBeTrue();
|
||||||
|
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
||||||
(await ScalarAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
(await ScalarAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
||||||
@@ -101,7 +106,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
|||||||
var s = SqlHistorianFixture.UniqueSchema();
|
var s = SqlHistorianFixture.UniqueSchema();
|
||||||
var tasks = Enumerable.Range(0, 4).Select(_ => Task.Run(() => Boot(Opts(s)).EnsureAsync(CancellationToken.None))).ToArray();
|
var tasks = Enumerable.Range(0, 4).Select(_ => Task.Run(() => Boot(Opts(s)).EnsureAsync(CancellationToken.None))).ToArray();
|
||||||
await Task.WhenAll(tasks);
|
await Task.WhenAll(tasks);
|
||||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.ModifiedTrailVersion);
|
||||||
}
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
@@ -110,7 +115,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
|||||||
fx.SkipIfUnavailable();
|
fx.SkipIfUnavailable();
|
||||||
var ex = await Should.ThrowAsync<HistorianSchemaMissingException>(
|
var ex = await Should.ThrowAsync<HistorianSchemaMissingException>(
|
||||||
() => Boot(Opts(SqlHistorianFixture.UniqueSchema(), auto: false)).EnsureAsync(CancellationToken.None));
|
() => Boot(Opts(SqlHistorianFixture.UniqueSchema(), auto: false)).EnsureAsync(CancellationToken.None));
|
||||||
ex.Message.ShouldContain("missing version 2");
|
ex.Message.ShouldContain("missing version 1 (found 0)");
|
||||||
}
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
|
|||||||
+181
@@ -0,0 +1,181 @@
|
|||||||
|
using Microsoft.Data.SqlClient;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.AlarmHistorian;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Gitea #674: a database left at schema version 1 (no Part 11 <c>SampleModified</c> trail) with
|
||||||
|
/// <c>AutoCreateSchema=false</c> keeps historizing. Before #674 the bootstrapper required v2 and refused EVERY call —
|
||||||
|
/// live writes, raw reads, alarm events, retention — until script 002 was applied by hand. Now only the features
|
||||||
|
/// that need the trail are refused (<c>BadHistoryOperationUnsupported</c>, nothing sent to SQL), and the
|
||||||
|
/// bootstrapper logs ONE warning for the whole provider. Built through <see cref="SqlServerHistorianProvider"/>, the
|
||||||
|
/// production wiring, so every seam shares the one bootstrapper the way a node does.
|
||||||
|
/// </summary>
|
||||||
|
[Collection(SqlHistorianCollection.Name)]
|
||||||
|
public sealed class SchemaV1PostureTests(SqlHistorianFixture fx)
|
||||||
|
{
|
||||||
|
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||||
|
private static readonly DateTime Recent = DateTime.UtcNow.AddHours(-1);
|
||||||
|
private static readonly HistoryUpdateContext Who = new("editor", "k1");
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_v1_schema_keeps_historizing_and_refuses_only_the_modified_trail_features()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var s = SqlHistorianFixture.UniqueSchema();
|
||||||
|
await ApplyV1Async(s);
|
||||||
|
var old = DateTime.UtcNow.AddDays(-40);
|
||||||
|
var loggers = new CapturingLoggerFactory();
|
||||||
|
var provider = new SqlServerHistorianProvider(new SqlServerHistorianOptions
|
||||||
|
{
|
||||||
|
ConnectionString = fx.ConnectionString, Schema = s, AutoCreateSchema = false, RetentionDays = 30,
|
||||||
|
});
|
||||||
|
var sp = new LoggerServices(loggers);
|
||||||
|
|
||||||
|
// Live writes (the continuous-historization drain) and raw reads work.
|
||||||
|
var writer = provider.CreateValueWriter(sp);
|
||||||
|
(await writer.WriteLiveValuesAsync("Plant.Speed",
|
||||||
|
[new HistorizationValue(old, HistorianValue.FromDouble(1), 0), new HistorizationValue(Recent, HistorianValue.FromDouble(2), 0)],
|
||||||
|
Ct)).ShouldBeTrue();
|
||||||
|
var source = provider.CreateDataSource(sp);
|
||||||
|
var raw = await source.ReadRawAsync("Plant.Speed", Recent.AddMinutes(-1), Recent.AddMinutes(1), 10, Ct);
|
||||||
|
raw.Samples.Select(v => v.Value).ShouldBe([2d]);
|
||||||
|
|
||||||
|
// Alarm events are written.
|
||||||
|
var alarms = provider.CreateAlarmWriter(sp);
|
||||||
|
(await alarms.WriteBatchAsync(
|
||||||
|
[new AlarmHistorianEvent("alarm-1", "Plant/Line/Eq", "HiTemp", "LimitAlarm", AlarmSeverity.High, "Activated", "m", "u", null, Recent)],
|
||||||
|
Ct)).ShouldBe([HistorianWriteOutcome.Ack]);
|
||||||
|
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Event")).ShouldBe(1);
|
||||||
|
|
||||||
|
// Modified reads: the interface's "no trail" signal (the node manager reports BadHistoryOperationUnsupported),
|
||||||
|
// not a backend failure — health stays clean.
|
||||||
|
await Should.ThrowAsync<NotSupportedException>(() =>
|
||||||
|
source.ReadModifiedAsync("Plant.Speed", Recent.AddDays(-1), Recent.AddDays(1), 10, null, Ct));
|
||||||
|
source.GetHealthSnapshot().LastError.ShouldBeNull();
|
||||||
|
|
||||||
|
// HistoryUpdate: every item refused, and NOTHING reaches SQL — a sent MERGE/DELETE would either change the
|
||||||
|
// samples or fail on the missing SampleModified target; an Insert on an unknown tag would create its Tag row.
|
||||||
|
var updater = provider.CreateUpdater(sp)!;
|
||||||
|
var tagsBefore = await CountAsync($"SELECT COUNT(*) FROM [{s}].Tag");
|
||||||
|
(await updater.UpdateDataAsync("Plant.Speed", HistoryUpdateMode.Update,
|
||||||
|
[new HistorizationValue(Recent, HistorianValue.FromDouble(99), 0), new HistorizationValue(Recent.AddSeconds(1), HistorianValue.FromDouble(98), 0)],
|
||||||
|
Who, Ct)).ShouldBe([HistorianStatusCodes.BadHistoryOperationUnsupported, HistorianStatusCodes.BadHistoryOperationUnsupported]);
|
||||||
|
(await updater.UpdateDataAsync("Plant.Brand.New", HistoryUpdateMode.Insert,
|
||||||
|
[new HistorizationValue(Recent, HistorianValue.FromDouble(1), 0)], Who, Ct))
|
||||||
|
.ShouldBe([HistorianStatusCodes.BadHistoryOperationUnsupported]);
|
||||||
|
(await updater.DeleteRawAsync("Plant.Speed", Recent.AddMinutes(-1), Recent.AddMinutes(1), Who, Ct))
|
||||||
|
.ShouldBe(HistorianStatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
(await updater.DeleteAtTimeAsync("Plant.Speed", [Recent], Who, Ct))
|
||||||
|
.ShouldBe([HistorianStatusCodes.BadHistoryOperationUnsupported]);
|
||||||
|
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Tag")).ShouldBe(tagsBefore);
|
||||||
|
(await source.ReadRawAsync("Plant.Speed", Recent.AddMinutes(-1), Recent.AddMinutes(1), 10, Ct))
|
||||||
|
.Samples.Select(v => v.Value).ShouldBe([2d]);
|
||||||
|
|
||||||
|
// Retention purges samples and skips the trail it does not have.
|
||||||
|
var purge = await provider.CreateRetention(sp)!.PurgeAsync(Ct);
|
||||||
|
purge.Select(r => r.Table).ShouldBe(["Sample", "Event"]);
|
||||||
|
purge.Single(r => r.Table == "Sample").Rows.ShouldBe(1);
|
||||||
|
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Sample")).ShouldBe(1);
|
||||||
|
|
||||||
|
// One warning for the whole provider (= per boot), however many seams and calls touched the bootstrapper.
|
||||||
|
loggers.Warnings.Count(w => w.Contains("Part 11 modification trail is unavailable")).ShouldBe(1);
|
||||||
|
(await CountAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(1); // verify-only: never upgraded
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: <c>AutoCreateSchema=true</c> still upgrades a v1 schema to v2 — the modification trail appears and a
|
||||||
|
/// HistoryUpdate is recorded in it — and logs no v1 warning.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Auto_create_still_upgrades_a_v1_schema_and_enables_the_trail()
|
||||||
|
{
|
||||||
|
fx.SkipIfUnavailable();
|
||||||
|
var s = SqlHistorianFixture.UniqueSchema();
|
||||||
|
await ApplyV1Async(s);
|
||||||
|
var loggers = new CapturingLoggerFactory();
|
||||||
|
var provider = new SqlServerHistorianProvider(new SqlServerHistorianOptions
|
||||||
|
{
|
||||||
|
ConnectionString = fx.ConnectionString, Schema = s, AutoCreateSchema = true,
|
||||||
|
});
|
||||||
|
var sp = new LoggerServices(loggers);
|
||||||
|
|
||||||
|
(await provider.CreateUpdater(sp)!.UpdateDataAsync("Plant.Speed", HistoryUpdateMode.Insert,
|
||||||
|
[new HistorizationValue(Recent, HistorianValue.FromDouble(5), 0)], Who, Ct))
|
||||||
|
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||||
|
|
||||||
|
(await CountAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.ModifiedTrailVersion);
|
||||||
|
var trail = await provider.CreateDataSource(sp)
|
||||||
|
.ReadModifiedAsync("Plant.Speed", Recent.AddMinutes(-1), Recent.AddMinutes(1), 10, null, Ct);
|
||||||
|
trail.Values.ShouldHaveSingleItem().UpdateType.ShouldBe(HistoryModificationType.Insert);
|
||||||
|
loggers.Warnings.ShouldNotContain(w => w.Contains("Part 11 modification trail is unavailable"));
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task ApplyV1Async(string schema)
|
||||||
|
{
|
||||||
|
var v1 = new StreamReader(typeof(SqlServerSchemaBootstrapper).Assembly.GetManifestResourceStream("Schema.001_initial.sql")!).ReadToEnd();
|
||||||
|
await using var c = new SqlConnection(fx.ConnectionString);
|
||||||
|
await c.OpenAsync(Ct);
|
||||||
|
foreach (var b in SqlServerSchemaBootstrapper.SplitBatches(v1.Replace("$(Schema)", schema)))
|
||||||
|
{
|
||||||
|
await using var cmd = c.CreateCommand();
|
||||||
|
cmd.CommandText = b;
|
||||||
|
await cmd.ExecuteNonQueryAsync(Ct);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<long> CountAsync(string sql)
|
||||||
|
{
|
||||||
|
await using var c = new SqlConnection(fx.ConnectionString);
|
||||||
|
await c.OpenAsync(Ct);
|
||||||
|
await using var cmd = c.CreateCommand();
|
||||||
|
cmd.CommandText = sql;
|
||||||
|
return Convert.ToInt64(await cmd.ExecuteScalarAsync(Ct));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>The only service the provider resolves is <see cref="ILoggerFactory"/>.</summary>
|
||||||
|
private sealed class LoggerServices(ILoggerFactory loggers) : IServiceProvider
|
||||||
|
{
|
||||||
|
public object? GetService(Type serviceType) => serviceType == typeof(ILoggerFactory) ? loggers : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Captures every Warning-or-above message from every logger it creates.</summary>
|
||||||
|
private sealed class CapturingLoggerFactory : ILoggerFactory
|
||||||
|
{
|
||||||
|
private readonly List<string> _warnings = [];
|
||||||
|
|
||||||
|
public IReadOnlyList<string> Warnings
|
||||||
|
{
|
||||||
|
get { lock (_warnings) return _warnings.ToList(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
public ILogger CreateLogger(string categoryName) => new Capturing(this);
|
||||||
|
|
||||||
|
public void AddProvider(ILoggerProvider provider)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class Capturing(CapturingLoggerFactory owner) : ILogger
|
||||||
|
{
|
||||||
|
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
|
||||||
|
|
||||||
|
public bool IsEnabled(LogLevel logLevel) => true;
|
||||||
|
|
||||||
|
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception,
|
||||||
|
Func<TState, Exception?, string> formatter)
|
||||||
|
{
|
||||||
|
if (logLevel < LogLevel.Warning) return;
|
||||||
|
lock (owner._warnings) owner._warnings.Add(formatter(state, exception));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+7
@@ -60,3 +60,10 @@ public sealed class SqlServerModifiedValueConformance(SqlHistorianFixture fx) :
|
|||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
|
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Collection(SqlHistorianCollection.Name)]
|
||||||
|
public sealed class SqlServerEventReadConformance(SqlHistorianFixture fx) : EventReadConformance
|
||||||
|
{
|
||||||
|
/// <inheritdoc/>
|
||||||
|
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
|
||||||
|
}
|
||||||
|
|||||||
+17
-1
@@ -34,12 +34,28 @@ internal sealed class SqlServerConformanceBackend : IConformanceBackend
|
|||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public IHistorianDataUpdater CreateUpdater() =>
|
public IHistorianDataUpdater CreateUpdater() =>
|
||||||
new SqlServerHistorianDataUpdater(_db.Conn, new SqlServerTagCatalog(_db.Conn, _db.Boot));
|
new SqlServerHistorianDataUpdater(_db.Conn, _db.Boot, new SqlServerTagCatalog(_db.Conn, _db.Boot));
|
||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public async Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples) =>
|
public async Task SeedAsync(string tag, params (DateTime T, HistorianValue V, uint Status)[] samples) =>
|
||||||
await _db.SeedAsync(tag, samples);
|
await _db.SeedAsync(tag, samples);
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public async Task SeedEventsAsync(params (DateTime T, string Id, string Source)[] events)
|
||||||
|
{
|
||||||
|
await using var c = await _db.Conn.OpenAsync(CancellationToken.None);
|
||||||
|
foreach (var (t, id, source) in events)
|
||||||
|
{
|
||||||
|
await using var cmd = _db.Conn.CreateCommand(c,
|
||||||
|
$"INSERT {_db.Conn.Q("Event")} (EventId, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, Message, " +
|
||||||
|
"UserName, Comment, EventTimeUtc, ReceivedTimeUtc) VALUES (@id, @src, 'a', 'n', 'T', 500, 'Activated', NULL, NULL, NULL, @t, @t)");
|
||||||
|
cmd.Parameters.AddWithValue("@id", id);
|
||||||
|
cmd.Parameters.AddWithValue("@src", source);
|
||||||
|
SqlHistorianConnection.AddDateTime2(cmd, "@t", t);
|
||||||
|
await cmd.ExecuteNonQueryAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public async Task<IReadOnlyList<(DateTime T, HistorianValue V, uint Status)>> ReadBackAsync(string tag)
|
public async Task<IReadOnlyList<(DateTime T, HistorianValue V, uint Status)>> ReadBackAsync(string tag)
|
||||||
{
|
{
|
||||||
|
|||||||
+3
-2
@@ -32,7 +32,7 @@ public sealed class SchemaScriptTests
|
|||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// #668: schema v2 (the Part 11 modification trail) is embedded, matches its docs copy, records version 2 (which
|
/// #668: schema v2 (the Part 11 modification trail) is embedded, matches its docs copy, records version 2 (which
|
||||||
/// must equal <see cref="SqlServerSchemaBootstrapper.RequiredVersion"/>), and carries none of the constructs an
|
/// must equal <see cref="SqlServerSchemaBootstrapper.ModifiedTrailVersion"/>), and carries none of the constructs an
|
||||||
/// <c>OUTPUT … INTO</c> target forbids (FOREIGN KEY / CHECK / trigger) — any of them would fail every HistoryUpdate.
|
/// <c>OUTPUT … INTO</c> target forbids (FOREIGN KEY / CHECK / trigger) — any of them would fail every HistoryUpdate.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
@@ -45,7 +45,8 @@ public sealed class SchemaScriptTests
|
|||||||
text.ShouldContain("[SampleModified]");
|
text.ShouldContain("[SampleModified]");
|
||||||
text.ShouldContain("PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc, ModifiedId)");
|
text.ShouldContain("PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc, ModifiedId)");
|
||||||
text.ShouldContain("VALUES (2, SYSUTCDATETIME())");
|
text.ShouldContain("VALUES (2, SYSUTCDATETIME())");
|
||||||
SqlServerSchemaBootstrapper.RequiredVersion.ShouldBe(2);
|
SqlServerSchemaBootstrapper.ModifiedTrailVersion.ShouldBe(2);
|
||||||
|
SqlServerSchemaBootstrapper.RequiredVersion.ShouldBe(1); // #674: v1 is still served, without the trail
|
||||||
var ddl = string.Join('\n', text.Split('\n').Where(l => !l.TrimStart().StartsWith("--", StringComparison.Ordinal)));
|
var ddl = string.Join('\n', text.Split('\n').Where(l => !l.TrimStart().StartsWith("--", StringComparison.Ordinal)));
|
||||||
ddl.ShouldNotContain("FOREIGN KEY", Case.Insensitive);
|
ddl.ShouldNotContain("FOREIGN KEY", Case.Insensitive);
|
||||||
ddl.ShouldNotContain("CHECK", Case.Insensitive);
|
ddl.ShouldNotContain("CHECK", Case.Insensitive);
|
||||||
|
|||||||
@@ -0,0 +1,78 @@
|
|||||||
|
using System.Text.Json;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Host.Tests.Configuration;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #678.6 — the per-role appsettings overlay name is built from plain roles only. A <c>cluster-*</c> role in the
|
||||||
|
/// name produced a file that never exists, so no overlay ever loaded on a node with a cluster role.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class RoleOverlayTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void A_driver_site_node_loads_the_driver_overlay()
|
||||||
|
=> RoleOverlay.Suffix(["driver", "cluster-SITE-A"]).ShouldBe("driver");
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void A_central_node_loads_the_admin_driver_overlay()
|
||||||
|
=> RoleOverlay.Suffix(["admin", "driver", "cluster-MAIN"]).ShouldBe("admin-driver");
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Plain_roles_are_ordinal_sorted_regardless_of_input_order()
|
||||||
|
=> RoleOverlay.Suffix(["driver", "cluster-MAIN", "admin"]).ShouldBe("admin-driver");
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Roles_without_a_cluster_role_keep_the_existing_names()
|
||||||
|
{
|
||||||
|
RoleOverlay.Suffix(["admin"]).ShouldBe("admin");
|
||||||
|
RoleOverlay.Suffix(["driver"]).ShouldBe("driver");
|
||||||
|
RoleOverlay.Suffix(["admin", "driver"]).ShouldBe("admin-driver");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void A_cluster_role_prefix_is_matched_case_insensitively()
|
||||||
|
=> RoleOverlay.Suffix(["driver", "CLUSTER-SITE-A"]).ShouldBe("driver");
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void No_plain_role_means_no_overlay()
|
||||||
|
{
|
||||||
|
RoleOverlay.Suffix([]).ShouldBeNull();
|
||||||
|
RoleOverlay.Suffix(["cluster-SITE-A"]).ShouldBeNull();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- The shipped overlays: activating them on cluster-role nodes must not change production semantics. ----
|
||||||
|
|
||||||
|
private static string HostDir()
|
||||||
|
{
|
||||||
|
for (var dir = new DirectoryInfo(AppContext.BaseDirectory); dir is not null; dir = dir.Parent)
|
||||||
|
{
|
||||||
|
var candidate = Path.Combine(dir.FullName, "src", "Server", "ZB.MOM.WW.OtOpcUa.Host");
|
||||||
|
if (File.Exists(Path.Combine(candidate, "appsettings.json"))) return candidate;
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new InvalidOperationException("Host project directory not found above " + AppContext.BaseDirectory);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData("admin")]
|
||||||
|
[InlineData("driver")]
|
||||||
|
[InlineData("admin-driver")]
|
||||||
|
public void Shipped_overlays_carry_only_the_audited_keys(string suffix)
|
||||||
|
{
|
||||||
|
using var doc = JsonDocument.Parse(File.ReadAllText(Path.Combine(HostDir(), $"appsettings.{suffix}.json")));
|
||||||
|
var root = doc.RootElement;
|
||||||
|
|
||||||
|
// Only Serilog + Security:Ldap, and in Security:Ldap only the two audited keys.
|
||||||
|
root.EnumerateObject().Select(p => p.Name).Order().ShouldBe(["Security", "Serilog"]);
|
||||||
|
var ldap = root.GetProperty("Security").GetProperty("Ldap");
|
||||||
|
ldap.EnumerateObject().Select(p => p.Name).Order().ShouldBe(["DevStubMode", "Transport"]);
|
||||||
|
ldap.GetProperty("DevStubMode").GetBoolean().ShouldBeFalse();
|
||||||
|
|
||||||
|
// No Debug-level (or lower) category override: activating an overlay must not raise log volume on a node.
|
||||||
|
foreach (var o in root.GetProperty("Serilog").GetProperty("MinimumLevel").GetProperty("Override").EnumerateObject())
|
||||||
|
o.Value.GetString().ShouldNotBeOneOf("Debug", "Verbose");
|
||||||
|
root.GetProperty("Serilog").GetProperty("MinimumLevel").GetProperty("Default").GetString().ShouldBe("Information");
|
||||||
|
}
|
||||||
|
}
|
||||||
+55
-3
@@ -54,8 +54,48 @@ public sealed class EventHistoryPagingOverTheWireTests
|
|||||||
finally { await rig.DisposeAsync(); }
|
finally { await rig.DisposeAsync(); }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_capped_REVERSE_event_read_pages_newest_first_through_the_session_store_exactly_once()
|
||||||
|
{
|
||||||
|
// Gitea #675: StartTime later than EndTime. Before the fix the cap kept the EARLIEST 10, ascending, with no
|
||||||
|
// continuation point.
|
||||||
|
var ct = TestContext.Current.CancellationToken;
|
||||||
|
var rig = await StartAsync("EvtPagingWireRev", encrypted: false, ct);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var events = Enumerable.Range(0, 25)
|
||||||
|
.Select(i => new HistoricalEvent($"e{i:00}", "src", Epoch.AddSeconds(i / 2), Epoch, "m", 500))
|
||||||
|
.ToList(); // pairs share a timestamp, so page boundaries cut through ties
|
||||||
|
rig.Nm.HistorianDataSource = new SqlLikeEventSource(events);
|
||||||
|
rig.MaterialiseNative();
|
||||||
|
rig.Enforce(false, [Key()]);
|
||||||
|
|
||||||
|
using var session = await rig.ConnectAsync(anonymous: true, ct);
|
||||||
|
var node = new NodeId(RawDeviceFolder, (ushort)session.NamespaceUris.GetIndex(V3NodeIds.RawNamespaceUri));
|
||||||
|
|
||||||
|
var collected = new List<string>();
|
||||||
|
var sizes = new List<int>();
|
||||||
|
byte[]? cp = null;
|
||||||
|
do
|
||||||
|
{
|
||||||
|
var (ids, next) = await ReadPageAsync(session, node, cp, ct, reverse: true);
|
||||||
|
collected.AddRange(ids);
|
||||||
|
sizes.Add(ids.Count);
|
||||||
|
cp = next;
|
||||||
|
sizes.Count.ShouldBeLessThan(30, "paging must terminate");
|
||||||
|
}
|
||||||
|
while (cp is not null);
|
||||||
|
|
||||||
|
sizes.ShouldBe([10, 10, 5]);
|
||||||
|
// Newest-first: time descending, ties by EventId descending.
|
||||||
|
collected.ShouldBe(events.OrderByDescending(e => e.EventTimeUtc)
|
||||||
|
.ThenByDescending(e => e.EventId, StringComparer.Ordinal).Select(e => e.EventId));
|
||||||
|
}
|
||||||
|
finally { await rig.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
private static async Task<(List<string> Ids, byte[]? Cp)> ReadPageAsync(
|
private static async Task<(List<string> Ids, byte[]? Cp)> ReadPageAsync(
|
||||||
ISession session, NodeId node, byte[]? cp, CancellationToken ct)
|
ISession session, NodeId node, byte[]? cp, CancellationToken ct, bool reverse = false)
|
||||||
{
|
{
|
||||||
var filter = new EventFilter();
|
var filter = new EventFilter();
|
||||||
filter.SelectClauses.Add(new SimpleAttributeOperand(ObjectTypeIds.BaseEventType, new QualifiedName("EventId"))
|
filter.SelectClauses.Add(new SimpleAttributeOperand(ObjectTypeIds.BaseEventType, new QualifiedName("EventId"))
|
||||||
@@ -64,8 +104,8 @@ public sealed class EventHistoryPagingOverTheWireTests
|
|||||||
});
|
});
|
||||||
var details = new ReadEventDetails
|
var details = new ReadEventDetails
|
||||||
{
|
{
|
||||||
StartTime = Epoch.AddDays(-1),
|
StartTime = reverse ? Epoch.AddDays(1) : Epoch.AddDays(-1),
|
||||||
EndTime = Epoch.AddDays(1),
|
EndTime = reverse ? Epoch.AddDays(-1) : Epoch.AddDays(1),
|
||||||
NumValuesPerNode = 10,
|
NumValuesPerNode = 10,
|
||||||
Filter = filter,
|
Filter = filter,
|
||||||
};
|
};
|
||||||
@@ -94,6 +134,18 @@ public sealed class EventHistoryPagingOverTheWireTests
|
|||||||
return new HistoricalEventsResult(window.Take(cap).ToList(), window.Count > cap ? new byte[] { 1 } : null);
|
return new HistoricalEventsResult(window.Take(cap).ToList(), window.Count > cap ? new byte[] { 1 } : null);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>The #675 overload: reverse is <c>(lo, hi]</c> by (time, id) descending.</summary>
|
||||||
|
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse, CancellationToken ct)
|
||||||
|
{
|
||||||
|
if (!reverse) return await ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, ct).ConfigureAwait(false);
|
||||||
|
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||||
|
var (lo, hi) = startUtc <= endUtc ? (startUtc, endUtc) : (endUtc, startUtc);
|
||||||
|
var window = _all.Where(e => e.EventTimeUtc > lo && e.EventTimeUtc <= hi).Reverse().ToList();
|
||||||
|
var cap = maxEvents <= 0 ? 1000 : maxEvents;
|
||||||
|
return new HistoricalEventsResult(window.Take(cap).ToList(), window.Count > cap ? new byte[] { 1 } : null);
|
||||||
|
}
|
||||||
|
|
||||||
public Task<CoreHistoryReadResult> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
public Task<CoreHistoryReadResult> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
||||||
NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct);
|
NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct);
|
||||||
public Task<CoreHistoryReadResult> ReadProcessedAsync(
|
public Task<CoreHistoryReadResult> ReadProcessedAsync(
|
||||||
|
|||||||
@@ -558,6 +558,190 @@ public sealed class HistoryUpdateTests : IDisposable
|
|||||||
finally { await host.DisposeAsync(); }
|
finally { await host.DisposeAsync(); }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ───────────────────────────── #674 future-skew bound ─────────────────────────────
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: an item whose SourceTimestamp lies beyond <c>now + HistoryUpdateMaxFutureSkew</c> is refused
|
||||||
|
/// <c>BadOutOfRange</c> by the node manager and never reaches the backend; an item exactly AT the bound and one in
|
||||||
|
/// the past are sent. Judged against the injected clock, not the wall clock.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task A_source_time_beyond_the_future_skew_is_bad_out_of_range_and_never_sent()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
nm.Clock = new FakeClock(Start);
|
||||||
|
nm.HistoryUpdateMaxFutureSkew = TimeSpan.FromHours(1);
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
var atBound = Start.AddHours(1);
|
||||||
|
|
||||||
|
var (results, errors) = Update(nm, KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate), nodeId,
|
||||||
|
PerformUpdateType.Update, Value(T1, 1d), Value(atBound.AddTicks(1), 2d), Value(atBound, 3d),
|
||||||
|
Value(new DateTime(2062, 1, 1, 0, 0, 0, DateTimeKind.Utc), 4d));
|
||||||
|
|
||||||
|
errors[0].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
results[0].OperationResults.Select(c => c.Code).ShouldBe(
|
||||||
|
[
|
||||||
|
StatusCodes.GoodEntryInserted, StatusCodes.BadOutOfRange, StatusCodes.GoodEntryInserted,
|
||||||
|
StatusCodes.BadOutOfRange,
|
||||||
|
]);
|
||||||
|
updater.UpdateCalls.ShouldHaveSingleItem().Values.Select(v => v.TimestampUtc).ShouldBe([T1, atBound]);
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>#674: every mode is bounded, and a request whose items are ALL out of range sends nothing.</summary>
|
||||||
|
[Theory]
|
||||||
|
[InlineData(PerformUpdateType.Insert)]
|
||||||
|
[InlineData(PerformUpdateType.Replace)]
|
||||||
|
[InlineData(PerformUpdateType.Update)]
|
||||||
|
public async Task Every_update_mode_is_bounded_and_an_all_refused_request_sends_nothing(PerformUpdateType type)
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
nm.Clock = new FakeClock(Start);
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
|
||||||
|
// Default skew: one day.
|
||||||
|
var (results, _) = Update(nm, KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate), nodeId, type,
|
||||||
|
Value(Start.AddDays(1).AddSeconds(1), 1d));
|
||||||
|
|
||||||
|
results[0].OperationResults.Select(c => c.Code).ShouldBe([StatusCodes.BadOutOfRange]);
|
||||||
|
updater.UpdateCalls.ShouldBeEmpty();
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>#674: a non-positive skew disables the bound — a far-future item reaches the backend.</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task A_non_positive_skew_disables_the_bound()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
nm.Clock = new FakeClock(Start);
|
||||||
|
nm.HistoryUpdateMaxFutureSkew = TimeSpan.Zero;
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
var farFuture = new DateTime(2062, 1, 1, 0, 0, 0, DateTimeKind.Utc);
|
||||||
|
|
||||||
|
var (results, _) = Insert(nm, KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate), nodeId, Value(farFuture, 1d));
|
||||||
|
|
||||||
|
results[0].OperationResults.Select(c => c.Code).ShouldBe([StatusCodes.GoodEntryInserted]);
|
||||||
|
updater.UpdateCalls.ShouldHaveSingleItem().Values.ShouldHaveSingleItem().TimestampUtc.ShouldBe(farFuture);
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
// ───────────────────────────── #674 Primary gate ─────────────────────────────
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: off-Primary, an authorized HistoryUpdate is refused <c>BadInvalidState</c> per node — on every arm
|
||||||
|
/// (update data, delete raw, delete at time) — and NOTHING reaches the updater.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Off_primary_every_arm_is_bad_invalid_state_and_nothing_is_sent()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
var consulted = 0;
|
||||||
|
nm.HistoryUpdatePrimaryGate = () => { Interlocked.Increment(ref consulted); return false; };
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
var identity = KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate);
|
||||||
|
|
||||||
|
var (results, errors) = Insert(nm, identity, nodeId, Value(T1, 1d));
|
||||||
|
var (rawResults, rawErrors) = Call(nm, identity, typeof(DeleteRawModifiedDetails),
|
||||||
|
new DeleteRawModifiedDetails { NodeId = nodeId, StartTime = T1, EndTime = T2 });
|
||||||
|
var (atResults, atErrors) = Call(nm, identity, typeof(DeleteAtTimeDetails),
|
||||||
|
new DeleteAtTimeDetails { NodeId = nodeId, ReqTimes = new DateTimeCollection { T1 } });
|
||||||
|
|
||||||
|
foreach (var (r, e) in new[] { (results, errors), (rawResults, rawErrors), (atResults, atErrors) })
|
||||||
|
{
|
||||||
|
e[0].StatusCode.Code.ShouldBe(StatusCodes.BadInvalidState);
|
||||||
|
r[0].StatusCode.Code.ShouldBe(StatusCodes.BadInvalidState);
|
||||||
|
}
|
||||||
|
|
||||||
|
consulted.ShouldBe(3);
|
||||||
|
updater.UpdateCalls.ShouldBeEmpty();
|
||||||
|
updater.DeleteRawCalls.ShouldBeEmpty();
|
||||||
|
updater.DeleteAtTimeCalls.ShouldBeEmpty();
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: on the Primary the gate allows and the update REACHES the updater — the allow leg that makes the deny
|
||||||
|
/// leg meaningful (a gate that refused everything, or one never consulted, fails one of the two).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task On_primary_the_update_reaches_the_updater()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
var consulted = 0;
|
||||||
|
nm.HistoryUpdatePrimaryGate = () => { Interlocked.Increment(ref consulted); return true; };
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
|
||||||
|
var (results, errors) = Insert(nm, KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate), nodeId, Value(T1, 1d));
|
||||||
|
|
||||||
|
errors[0].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
results[0].OperationResults.Select(c => c.Code).ShouldBe([StatusCodes.GoodEntryInserted]);
|
||||||
|
updater.UpdateCalls.ShouldHaveSingleItem().Tag.ShouldBe(Tagname);
|
||||||
|
consulted.ShouldBe(1);
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: the Primary gate runs AFTER authorization — an unauthorized caller is told <c>BadUserAccessDenied</c>
|
||||||
|
/// (not the node's role) and the gate is never consulted.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task The_primary_gate_runs_after_authorization()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var updater = new FakeUpdater();
|
||||||
|
nm.HistorianDataUpdater = updater;
|
||||||
|
var consulted = 0;
|
||||||
|
nm.HistoryUpdatePrimaryGate = () => { Interlocked.Increment(ref consulted); return false; };
|
||||||
|
WireAcl(nm, enabled: false, NodePermissions.WriteOperate);
|
||||||
|
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||||
|
|
||||||
|
var (results, _) = Insert(nm, KeyIdentity(OpcUaDataPlaneRoles.WriteOperate), nodeId, Value(T1, 1d));
|
||||||
|
|
||||||
|
results[0].StatusCode.Code.ShouldBe(StatusCodes.BadUserAccessDenied);
|
||||||
|
consulted.ShouldBe(0);
|
||||||
|
updater.UpdateCalls.ShouldBeEmpty();
|
||||||
|
}
|
||||||
|
finally { await host.DisposeAsync(); }
|
||||||
|
}
|
||||||
|
|
||||||
// ───────────────────────────── helpers ─────────────────────────────
|
// ───────────────────────────── helpers ─────────────────────────────
|
||||||
|
|
||||||
private static DataValue Value(DateTime timestamp, object value) =>
|
private static DataValue Value(DateTime timestamp, object value) =>
|
||||||
|
|||||||
+186
-7
@@ -199,11 +199,168 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
|
|||||||
|
|
||||||
var mixed = ReadBatch(nm, 4, null, (node, good1[0].Cp), (other, Guid.NewGuid().ToByteArray()));
|
var mixed = ReadBatch(nm, 4, null, (node, good1[0].Cp), (other, Guid.NewGuid().ToByteArray()));
|
||||||
mixed[0].Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
mixed[0].Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
mixed[0].Ids.ShouldBe(src.All.Skip(4).Take(3).Select(e => e.EventId));
|
// A resumed page is FULL (#675): it over-fetches by the boundary skip it then trims.
|
||||||
|
mixed[0].Ids.ShouldBe(src.All.Skip(4).Take(4).Select(e => e.EventId));
|
||||||
mixed[1].Error.StatusCode.Code.ShouldBe(StatusCodes.BadContinuationPointInvalid);
|
mixed[1].Error.StatusCode.Code.ShouldBe(StatusCodes.BadContinuationPointInvalid);
|
||||||
await host.DisposeAsync();
|
await host.DisposeAsync();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- Gitea #675: reverse (newest-first) event paging ------------------------------------------------------
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_events_page_newest_first_exactly_once_with_exact_page_counts()
|
||||||
|
{
|
||||||
|
var (host, nm, node, src) = await SetupAsync(Distinct(250));
|
||||||
|
var pages = new List<int>();
|
||||||
|
var ids = DrainAsync(nm, node, max: 100, start: Late, end: Early, pageSizes: pages);
|
||||||
|
ids.ShouldBe(src.All.Reverse().Select(e => e.EventId));
|
||||||
|
pages.ShouldBe([100, 100, 50]);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_first_page_is_the_LATEST_events_with_a_continuation_point()
|
||||||
|
{
|
||||||
|
// The pre-#675 shape: the cap kept the EARLIEST N, ascending, Good, and no continuation point.
|
||||||
|
var (host, nm, node, _) = await SetupAsync(Distinct(10));
|
||||||
|
var p1 = ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late);
|
||||||
|
p1.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
p1.Ids.ShouldBe(["e0009", "e0008", "e0007"]);
|
||||||
|
p1.Cp.ShouldNotBeNull("a capped reverse read has more behind it and must page");
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_tie_at_page_boundary_is_neither_duplicated_nor_skipped()
|
||||||
|
{
|
||||||
|
var events = new List<HistoricalEvent>
|
||||||
|
{
|
||||||
|
Ev("e0", 0), Ev("e1", 1), Ev("e2", 2), Ev("e3", 2), Ev("e4", 2), Ev("e5", 3),
|
||||||
|
};
|
||||||
|
var (host, nm, node, _) = await SetupAsync(events);
|
||||||
|
// Newest-first, ties in descending EventId: cap 2 cuts the cluster at e4|e3.
|
||||||
|
DrainAsync(nm, node, max: 2, start: Late, end: Early).ShouldBe(["e5", "e4", "e3", "e2", "e1", "e0"]);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_tie_cluster_larger_than_the_cap_pages_within_the_cluster()
|
||||||
|
{
|
||||||
|
var events = new List<HistoricalEvent> { Ev("a0", 0) };
|
||||||
|
for (var i = 0; i < 7; i++) events.Add(Ev($"t{i}", 5));
|
||||||
|
events.Add(Ev("z0", 9));
|
||||||
|
var (host, nm, node, _) = await SetupAsync(events);
|
||||||
|
var pages = new List<int>();
|
||||||
|
DrainAsync(nm, node, max: 3, start: Late, end: Early, pageSizes: pages).ShouldBe(
|
||||||
|
["z0", "t6", "t5", "t4", "t3", "t2", "t1", "t0", "a0"]);
|
||||||
|
pages.Sum().ShouldBe(9);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_tie_cluster_pages_on_a_coarse_resolution_backend()
|
||||||
|
{
|
||||||
|
const long res = 10;
|
||||||
|
var events = new List<HistoricalEvent>();
|
||||||
|
DateTime At(int slot) => Epoch.AddTicks(slot * res);
|
||||||
|
events.Add(new("a0", "s", At(0), Epoch, "m", 500));
|
||||||
|
for (var i = 0; i < 7; i++) events.Add(new($"t{i}", "s", At(3), Epoch, "m", 500));
|
||||||
|
events.Add(new("z0", "s", At(8), Epoch, "m", 500));
|
||||||
|
var (host, nm, node, _) = await SetupAsync(events, res);
|
||||||
|
DrainAsync(nm, node, max: 3, start: Late, end: Early).ShouldBe(
|
||||||
|
["z0", "t6", "t5", "t4", "t3", "t2", "t1", "t0", "a0"]);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_tie_cluster_beyond_the_overfetch_bound_fails_loudly()
|
||||||
|
{
|
||||||
|
var events = new List<HistoricalEvent>();
|
||||||
|
for (var i = 0; i < 10; i++) events.Add(Ev($"t{i:00}", 5));
|
||||||
|
var (host, nm, node, _) = await SetupAsync(events);
|
||||||
|
nm.MaxTieClusterOverfetch = 4;
|
||||||
|
var p1 = ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late);
|
||||||
|
p1.Cp.ShouldNotBeNull();
|
||||||
|
ReadEvents(nm, node, max: 3, inboundCp: p1.Cp, end: Early, start: Late).Error.StatusCode.Code
|
||||||
|
.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_reverse_cursor_resumes_reverse_whatever_the_resume_request_says()
|
||||||
|
{
|
||||||
|
// The direction lives in the cursor: a resume carrying FORWARD details still continues newest-first.
|
||||||
|
var (host, nm, node, _) = await SetupAsync(Distinct(10));
|
||||||
|
var p1 = ReadEvents(nm, node, max: 4, inboundCp: null, end: Early, start: Late);
|
||||||
|
p1.Ids.ShouldBe(["e0009", "e0008", "e0007", "e0006"]);
|
||||||
|
var p2 = ReadEvents(nm, node, max: 4, inboundCp: p1.Cp); // default details: Early -> Late (forward)
|
||||||
|
p2.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
p2.Ids.ShouldBe(["e0005", "e0004", "e0003", "e0002"]);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_forward_cursor_resumes_forward_whatever_the_resume_request_says()
|
||||||
|
{
|
||||||
|
var (host, nm, node, _) = await SetupAsync(Distinct(10));
|
||||||
|
var p1 = ReadEvents(nm, node, max: 4, inboundCp: null);
|
||||||
|
p1.Ids.ShouldBe(["e0000", "e0001", "e0002", "e0003"]);
|
||||||
|
var p2 = ReadEvents(nm, node, max: 4, inboundCp: p1.Cp, end: Early, start: Late);
|
||||||
|
p2.Ids.ShouldBe(["e0004", "e0005", "e0006", "e0007"]);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task Reverse_unbounded_read_returns_everything_newest_first_without_paging()
|
||||||
|
{
|
||||||
|
var (host, nm, node, src) = await SetupAsync(Distinct(30));
|
||||||
|
var r = ReadEvents(nm, node, max: 0, inboundCp: null, end: Early, start: Late);
|
||||||
|
r.Ids.ShouldBe(src.All.Reverse().Select(e => e.EventId));
|
||||||
|
r.Cp.ShouldBeNull();
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task End_only_events_read_pages_backward_from_EndTime()
|
||||||
|
{
|
||||||
|
// StartTime unset + EndTime + NumValuesPerNode > 0 => backward from EndTime (Part 11; mirrors raw). The reverse
|
||||||
|
// window is (.., EndTime], so an event exactly AT EndTime is the first one returned.
|
||||||
|
var (host, nm, node, _) = await SetupAsync(Distinct(25));
|
||||||
|
var ids = DrainAsync(nm, node, max: 10, start: DateTime.MinValue, end: Epoch.AddSeconds(20));
|
||||||
|
ids.ShouldBe(Enumerable.Range(0, 21).Reverse().Select(i => $"e{i:0000}"));
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task A_backend_without_reverse_support_answers_BadHistoryOperationUnsupported_not_the_earliest_events()
|
||||||
|
{
|
||||||
|
var (host, nm, node, src) = await SetupAsync(Distinct(10));
|
||||||
|
nm.HistorianDataSource = new ForwardOnlyEventSource(src);
|
||||||
|
ReadEvents(nm, node, max: 3, inboundCp: null, end: Early, start: Late).Error.StatusCode.Code
|
||||||
|
.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
// Forward reads on the same backend are unaffected.
|
||||||
|
DrainAsync(nm, node, max: 4).ShouldBe(src.All.Select(e => e.EventId));
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static readonly DateTime Early = Epoch.AddDays(-1);
|
||||||
|
private static readonly DateTime Late = Epoch.AddDays(1);
|
||||||
|
|
||||||
|
/// <summary>A backend that implements only the five-argument (forward) event read - Gateway / Null shaped.</summary>
|
||||||
|
private sealed class ForwardOnlyEventSource(IHistorianDataSource inner) : IHistorianDataSource
|
||||||
|
{
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(string? n, DateTime s, DateTime e, int m, CancellationToken ct) =>
|
||||||
|
inner.ReadEventsAsync(n, s, e, m, ct);
|
||||||
|
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
||||||
|
inner.ReadRawAsync(r, s, e, m, ct);
|
||||||
|
public Task<HistorianRead> ReadProcessedAsync(string r, DateTime s, DateTime e, TimeSpan i, HistoryAggregateType a, CancellationToken ct) =>
|
||||||
|
inner.ReadProcessedAsync(r, s, e, i, a, ct);
|
||||||
|
public Task<HistorianRead> ReadAtTimeAsync(string r, IReadOnlyList<DateTime> t, CancellationToken ct) =>
|
||||||
|
inner.ReadAtTimeAsync(r, t, ct);
|
||||||
|
public HistorianHealthSnapshot GetHealthSnapshot() => inner.GetHealthSnapshot();
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
private sealed class RawSeriesSource : IHistorianDataSource
|
private sealed class RawSeriesSource : IHistorianDataSource
|
||||||
{
|
{
|
||||||
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct)
|
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct)
|
||||||
@@ -265,16 +422,18 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
|
|||||||
return (started.Host, nm, nm.TryGetFolder(equipmentId)!.NodeId, src);
|
return (started.Host, nm, nm.TryGetFolder(equipmentId)!.NodeId, src);
|
||||||
}
|
}
|
||||||
|
|
||||||
private static List<string> DrainAsync(OtOpcUaNodeManager nm, NodeId node, uint max, DateTime? end = null)
|
private static List<string> DrainAsync(
|
||||||
|
OtOpcUaNodeManager nm, NodeId node, uint max, DateTime? end = null, DateTime? start = null, List<int>? pageSizes = null)
|
||||||
{
|
{
|
||||||
var all = new List<string>();
|
var all = new List<string>();
|
||||||
byte[]? cp = null;
|
byte[]? cp = null;
|
||||||
var pages = 0;
|
var pages = 0;
|
||||||
do
|
do
|
||||||
{
|
{
|
||||||
var r = ReadEvents(nm, node, max, cp, end);
|
var r = ReadEvents(nm, node, max, cp, end, start);
|
||||||
r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
all.AddRange(r.Ids);
|
all.AddRange(r.Ids);
|
||||||
|
pageSizes?.Add(r.Ids.Count);
|
||||||
cp = r.Cp;
|
cp = r.Cp;
|
||||||
(++pages).ShouldBeLessThan(60, "paging must terminate");
|
(++pages).ShouldBeLessThan(60, "paging must terminate");
|
||||||
}
|
}
|
||||||
@@ -283,11 +442,15 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
|
|||||||
}
|
}
|
||||||
|
|
||||||
private static (List<string> Ids, ServiceResult Error, byte[]? Cp) ReadEvents(
|
private static (List<string> Ids, ServiceResult Error, byte[]? Cp) ReadEvents(
|
||||||
OtOpcUaNodeManager nm, NodeId node, uint max, byte[]? inboundCp, DateTime? end = null) =>
|
OtOpcUaNodeManager nm, NodeId node, uint max, byte[]? inboundCp, DateTime? end = null, DateTime? start = null) =>
|
||||||
ReadBatch(nm, max, end, (node, inboundCp))[0];
|
ReadBatch(nm, max, end, start, (node, inboundCp))[0];
|
||||||
|
|
||||||
private static List<(List<string> Ids, ServiceResult Error, byte[]? Cp)> ReadBatch(
|
private static List<(List<string> Ids, ServiceResult Error, byte[]? Cp)> ReadBatch(
|
||||||
OtOpcUaNodeManager nm, uint max, DateTime? end, params (NodeId Node, byte[]? Cp)[] reads)
|
OtOpcUaNodeManager nm, uint max, DateTime? end, params (NodeId Node, byte[]? Cp)[] reads) =>
|
||||||
|
ReadBatch(nm, max, end, null, reads);
|
||||||
|
|
||||||
|
private static List<(List<string> Ids, ServiceResult Error, byte[]? Cp)> ReadBatch(
|
||||||
|
OtOpcUaNodeManager nm, uint max, DateTime? end, DateTime? start, params (NodeId Node, byte[]? Cp)[] reads)
|
||||||
{
|
{
|
||||||
var context = new OperationContext(
|
var context = new OperationContext(
|
||||||
new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
|
new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
|
||||||
@@ -298,7 +461,7 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
|
|||||||
});
|
});
|
||||||
var details = new ReadEventDetails
|
var details = new ReadEventDetails
|
||||||
{
|
{
|
||||||
StartTime = Epoch.AddDays(-1),
|
StartTime = start ?? Epoch.AddDays(-1),
|
||||||
EndTime = end ?? Epoch.AddDays(1),
|
EndTime = end ?? Epoch.AddDays(1),
|
||||||
NumValuesPerNode = max,
|
NumValuesPerNode = max,
|
||||||
Filter = filter,
|
Filter = filter,
|
||||||
@@ -361,6 +524,22 @@ public sealed class NodeManagerHistoryReadEventsPagingTests : IDisposable
|
|||||||
window.Take(cap).ToList(), truncated ? new byte[] { 1 } : null));
|
window.Take(cap).ToList(), truncated ? new byte[] { 1 } : null));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>Like the SQL backends' #675 overload: reverse => <c>(lo, hi]</c> newest-first by (time, id) descending.</summary>
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents, bool reverse, CancellationToken ct)
|
||||||
|
{
|
||||||
|
if (!reverse) return ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, ct);
|
||||||
|
Calls++;
|
||||||
|
var (lo, hi) = startUtc <= endUtc ? (startUtc, endUtc) : (endUtc, startUtc);
|
||||||
|
lo = Trunc(lo);
|
||||||
|
hi = Trunc(hi);
|
||||||
|
var window = All.Where(e => e.EventTimeUtc > lo && e.EventTimeUtc <= hi).Reverse().ToList();
|
||||||
|
var cap = maxEvents <= 0 ? 1000 : maxEvents;
|
||||||
|
var truncated = window.Count > cap;
|
||||||
|
return Task.FromResult(new HistoricalEventsResult(
|
||||||
|
window.Take(cap).ToList(), truncated ? new byte[] { 1 } : null));
|
||||||
|
}
|
||||||
|
|
||||||
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
||||||
NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct);
|
NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct);
|
||||||
public Task<HistorianRead> ReadProcessedAsync(
|
public Task<HistorianRead> ReadProcessedAsync(
|
||||||
|
|||||||
+22
@@ -203,6 +203,28 @@ public sealed class NodeManagerHistoryReadModifiedTests : IDisposable
|
|||||||
await host.DisposeAsync();
|
await host.DisposeAsync();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: the HistoryUpdate Primary gate does not reach HistoryRead — a closed gate neither refuses a read nor is
|
||||||
|
/// consulted by one (reads are served from any redundancy node), for modified and raw reads alike.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task The_history_update_primary_gate_is_not_consulted_for_reads()
|
||||||
|
{
|
||||||
|
var (host, nm, node, src) = await SetupAsync([Mod(1, 1, 1, HistoryModificationType.Update, "u", Epoch)]);
|
||||||
|
var consulted = 0;
|
||||||
|
nm.HistoryUpdatePrimaryGate = () => { Interlocked.Increment(ref consulted); return false; };
|
||||||
|
|
||||||
|
var modified = Read(nm, node, Epoch, Epoch.AddHours(1), max: 10, cp: null);
|
||||||
|
var raw = Read(nm, node, Epoch, Epoch.AddHours(1), max: 10, cp: null, modified: false);
|
||||||
|
|
||||||
|
modified.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
modified.Data!.DataValues.Count.ShouldBe(1);
|
||||||
|
raw.Error.StatusCode.Code.ShouldNotBe(StatusCodes.BadInvalidState);
|
||||||
|
src.Calls.ShouldBe(1);
|
||||||
|
consulted.ShouldBe(0);
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task A_backend_failure_is_isolated_to_its_node()
|
public async Task A_backend_failure_is_isolated_to_its_node()
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -165,6 +165,7 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
|||||||
StartTime = DateTime.UtcNow.AddHours(-1),
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
EndTime = DateTime.UtcNow,
|
EndTime = DateTime.UtcNow,
|
||||||
NumValuesPerNode = 10,
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false, // SDK default is true; this test is about the raw path
|
||||||
};
|
};
|
||||||
|
|
||||||
var (_, errors) = InvokeHistoryRead(server, nm, details, nodeId);
|
var (_, errors) = InvokeHistoryRead(server, nm, details, nodeId);
|
||||||
|
|||||||
+22
-2
@@ -223,10 +223,30 @@ public sealed class DriverHostActorHistorizationRoleViewTests : RuntimeActorTest
|
|||||||
host.Tell(new ClusterEvent.UnreachableMember(MemberAt(Peer672)));
|
host.Tell(new ClusterEvent.UnreachableMember(MemberAt(Peer672)));
|
||||||
AwaitAssert(() => hist.Published.ShouldBe([false, true]), duration: Timeout);
|
AwaitAssert(() => hist.Published.ShouldBe([false, true]), duration: Timeout);
|
||||||
|
|
||||||
// And self "returning" does not close what the peer's loss opened.
|
// Self "returning" is ignored too, and it does not close what the peer's loss opened. Events are
|
||||||
|
// processed in order, so the peer's return (which MUST close the gate) is the only thing that may
|
||||||
|
// appear next: had the self event published anything it would sit between `true` and `false`.
|
||||||
host.Tell(new ClusterEvent.ReachableMember(MemberAt(Self672)));
|
host.Tell(new ClusterEvent.ReachableMember(MemberAt(Self672)));
|
||||||
|
host.Tell(new ClusterEvent.ReachableMember(MemberAt(Peer672)));
|
||||||
|
AwaitAssert(() => hist.Published.ShouldBe([false, true, false]), duration: Timeout);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A host that starts while its peer is ALREADY unreachable (the <c>InitialStateAsEvents</c> replay
|
||||||
|
/// delivers the <c>UnreachableMember</c> before the first redundancy snapshot) must still treat that
|
||||||
|
/// peer as unreachable when the snapshot naming it Primary finally arrives.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void A_peer_unreachable_before_the_first_snapshot_still_opens_capture()
|
||||||
|
{
|
||||||
|
var hist = new RecordingHistorizationView();
|
||||||
|
IActorRef host = SpawnHost672(hist, others: null, captureWhilePrimaryUnreachable: true);
|
||||||
|
|
||||||
|
host.Tell(new ClusterEvent.UnreachableMember(MemberAt(Peer672)));
|
||||||
host.Tell(Snapshot((Self672, RedundancyRole.Secondary), (Peer672, RedundancyRole.Primary)));
|
host.Tell(Snapshot((Self672, RedundancyRole.Secondary), (Peer672, RedundancyRole.Primary)));
|
||||||
AwaitAssert(() => hist.Published.ShouldBe([false, true, true]), duration: Timeout);
|
|
||||||
|
// Exactly one publication, and it is `true`: no spurious `false` from treating the peer as reachable.
|
||||||
|
AwaitAssert(() => hist.Published.ShouldBe([true]), duration: Timeout);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary><c>CaptureWhilePrimaryUnreachable = false</c> restores the pre-#672 behaviour.</summary>
|
/// <summary><c>CaptureWhilePrimaryUnreachable = false</c> restores the pre-#672 behaviour.</summary>
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
using Xunit;
|
using Xunit;
|
||||||
using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||||
|
|
||||||
@@ -99,6 +100,29 @@ public sealed class ServerHistorianOptionsTests
|
|||||||
Assert.Contains(w, m => m.Contains("MaxTimeWeightedRawRows"));
|
Assert.Contains(w, m => m.Contains("MaxTimeWeightedRawRows"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// #674: the HistoryUpdate future-skew bound defaults to one day and binds from configuration as a TimeSpan;
|
||||||
|
/// a non-positive value is a deliberate "disabled", not a misconfiguration, so it does not warn.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void HistoryUpdateMaxFutureSkew_defaults_to_one_day_binds_and_zero_is_silent()
|
||||||
|
{
|
||||||
|
Assert.Equal(TimeSpan.FromDays(1), new ServerHistorianOptions().HistoryUpdateMaxFutureSkew);
|
||||||
|
var bound = new ConfigurationBuilder()
|
||||||
|
.AddInMemoryCollection(new Dictionary<string, string?>
|
||||||
|
{
|
||||||
|
["ServerHistorian:HistoryUpdateMaxFutureSkew"] = "00:05:00",
|
||||||
|
})
|
||||||
|
.Build()
|
||||||
|
.GetSection(ServerHistorianOptions.SectionName)
|
||||||
|
.Get<ServerHistorianOptions>();
|
||||||
|
Assert.Equal(TimeSpan.FromMinutes(5), bound!.HistoryUpdateMaxFutureSkew);
|
||||||
|
var w = new ServerHistorianOptions
|
||||||
|
{ Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", HistoryUpdateMaxFutureSkew = TimeSpan.Zero }
|
||||||
|
.Validate();
|
||||||
|
Assert.Empty(w);
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public void Disabled_with_nonpositive_HistoryRead_knobs_is_silent()
|
public void Disabled_with_nonpositive_HistoryRead_knobs_is_silent()
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user