feat(historian): Part 11 modified-value history on SqlServer + QuestDB (#668)
Every HistoryUpdate a SQL backend applies now leaves a modification trail
(prior value, principal, Part 11 update type, modification time), and
HistoryReadRaw with IsReadModified=true serves it as HistoryModifiedData.
- Core: HistoryUpdateContext threaded into all three IHistorianDataUpdater
methods; IHistorianDataSource.ReadModifiedAsync (default NotSupported =>
BadHistoryOperationUnsupported on Gateway/Null); HistoryModified* contracts.
- SqlServer: schema v2 (002_sample_modified.sql, RequiredVersion=2, docs copy);
trail written by OUTPUT ... INTO on the same MERGE/DELETE statement, so a
failed update leaves no audit row; retention purges SampleModified per tag on
the source-time cutoff; ReadModifiedAsync keyed on (SourceTimeUtc, ModifiedId).
- QuestDB: append-only {prefix}sample_modified (TTL like samples), trail row
written BEFORE the change (orphan-on-crash, never an untracked change);
existence reads now carry the prior value; ReadModifiedAsync.
- Node manager: modified reads dispatch + page via the #669 continuation store
(HistoryContinuationState gains HistoryCursorKind + LastModifiedId), gated by
the existing HistoryRead decision; session principal / key id passed to the
updater. IsDeleteModified stays unsupported.
- Client.CLI: historyread --modified.
- Tests: node-manager dispatch/paging/cursor binding/principal threading,
over-the-wire HistoryModifiedData, shared ModifiedValueConformance on both
backends, SqlServer atomicity (forced trail failure => no change) + v1->v2
upgrade + retention trail purge.
- Docs: Historian.md (Modified-value history section, schema v2 upgrade note),
security.md, Client.CLI.md, CLAUDE.md.
This commit is contained in:
Joseph Doherty
@@ -1346,6 +1346,7 @@ historian tagname, which is the tag's driver `FullName`.
|
||||
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
|
||||
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
|
||||
- **HistoryUpdate** (SqlServer only; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`.
|
||||
- **Modified-value history (#668, Part 11):** every applied HistoryUpdate item leaves a trail row (prior value, principal from the session's API key / `anonymous`, `HistoryModificationType`, time) — SqlServer `hist.SampleModified` (**schema v2**, written by `OUTPUT … INTO` in the same statement ⇒ atomic; ⚠️ no FK/CHECK/trigger on it, and `AutoCreateSchema=false` installs must run `002_sample_modified.sql` before upgrading), QuestDB `{p}sample_modified` (append-only, written **before** the change ⇒ a crash leaves an orphan trail row, never an untracked change). `IsReadModified=true` HistoryRead serves it as `HistoryModifiedData` via `IHistorianDataSource.ReadModifiedAsync` (default ⇒ `NotSupported` ⇒ `BadHistoryOperationUnsupported` on Gateway/Null), paged on `(SourceTime, ModifiedId)`, gated by plain HistoryRead. Live writes and retention never write it; retention/TTL purge it on source time. `IHistorianDataUpdater` methods take a `HistoryUpdateContext`. CLI `historyread --modified`. `docs/Historian.md` §Modified-value history.
|
||||
|
||||
### Read path (`ServerHistorian` section)
|
||||
|
||||
|
||||
@@ -210,8 +210,19 @@ otopcua-cli historyread -u opc.tcp://localhost:4840/OtOpcUa \
|
||||
-n "ns=1;s=TestMachine_001.TestHistoryValue" \
|
||||
--start "2026-03-25T00:00:00Z" --end "2026-03-30T00:00:00Z" \
|
||||
--aggregate Average --interval 3600000
|
||||
|
||||
# Modified-value history (what HistoryUpdate changed): prior value, update type, user, modification time
|
||||
otopcua-cli historyread -u opc.tcp://localhost:4840/OtOpcUa -U apikey -P otua_<keyId>_<secret> \
|
||||
-n "ns=2;s=Plant/Modbus/dev1/Speed" \
|
||||
--start "2026-01-15T00:00:00Z" --end "2026-01-16T00:00:00Z" --modified
|
||||
```
|
||||
|
||||
`--modified` sends `ReadRawModifiedDetails` with `IsReadModified = true` (OPC UA Part 11, Gitea #668) and prints one line per
|
||||
recorded modification: the source timestamp, the **prior** value (`(none)` for an insert, which had none), its status, the update type
|
||||
(`Insert` / `Replace` / `Update` / `Delete`), the user (the API key's principal, or `anonymous`) and the modification time. It follows
|
||||
continuation points up to `--max`. A server on the `Gateway` backend (no modification trail) answers `BadHistoryOperationUnsupported`.
|
||||
`--modified` cannot be combined with `--aggregate`. See [Historian.md §Modified-value history](Historian.md#modified-value-history-part-11).
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `-n` / `--node` | Node ID to read history for (required) |
|
||||
@@ -220,6 +231,7 @@ otopcua-cli historyread -u opc.tcp://localhost:4840/OtOpcUa \
|
||||
| `--max` | Maximum number of values (default: 1000) |
|
||||
| `--aggregate` | Aggregate function: Average, Minimum, Maximum, Count, Start, End, StandardDeviation |
|
||||
| `--interval` | Processing interval in milliseconds for aggregates (default: 3600000) |
|
||||
| `--modified` | Read modified-value history (Part 11 `IsReadModified`) instead of raw samples |
|
||||
|
||||
### historyupdate
|
||||
|
||||
|
||||
+86
-28
@@ -73,12 +73,13 @@ consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `A
|
||||
|
||||
| | `Gateway` (default) | `SqlServer` | `QuestDb` |
|
||||
|---|---|---|---|
|
||||
| Storage | AVEVA Historian via the HistorianGateway sidecar | `{Schema}` tables in a SQL Server database you point it at | `{prefix}sample` / `event` / `tag` WAL tables in a QuestDB instance |
|
||||
| Storage | AVEVA Historian via the HistorianGateway sidecar | `{Schema}` tables in a SQL Server database you point it at | `{prefix}sample` / `event` / `tag` / `sample_modified` WAL tables in a QuestDB instance |
|
||||
| Value kinds recorded | `Double`, `Int64`, `Boolean` (numeric analog only) | all: `Double`, `Int64`, `Boolean`, `String`, `DateTime`, plus `Empty` (a Bad sample with no value) | same as `SqlServer` |
|
||||
| Status code stored | Sent as DA quality but **discarded** by the gateway (C-002); AVEVA stores every sample Good | Stored bit-for-bit (`StatusCode` int column) | Stored bit-for-bit (unsigned, in a `LONG`) |
|
||||
| Comms-loss / driver-Bad publishes | Dropped, counted in `DroppedUnsupportedKind` | Stored as an `Empty` outage-marker sample carrying the Bad code (see [below](#typed-values-real-status-and-outage-markers)) | Stored as an `Empty` outage-marker sample, as on `SqlServer` |
|
||||
| Processed aggregates | Average, Minimum, Maximum, Total, Count | The thirteen in the [aggregate table](#aggregates-processed-reads) | The thirteen, pushed down to QuestDB SQL |
|
||||
| HistoryUpdate | No (`BadHistoryOperationUnsupported`) | Yes (Insert / Replace / Update / DeleteRaw / DeleteAtTime) | Yes — soft delete (tombstones), see [below](#historyupdate-soft-delete) |
|
||||
| Modified-value history (`IsReadModified`, #668) | No (`BadHistoryOperationUnsupported`) | Yes — `SampleModified`, atomic with the change | Yes — `{prefix}sample_modified`, written before the change (orphan-on-crash) |
|
||||
| Alarm history | gateway `SendEvent` | `Event` table (insert-if-absent) | `{prefix}event` table (dedup upsert on `(ts, event_id)`) |
|
||||
| Provisioning (`EnsureTags`) | numeric types only; others skipped | `MERGE` upsert of every request, all types | Upsert into `{prefix}tag`, all types |
|
||||
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` | `ServerHistorian:QuestDb:*` |
|
||||
@@ -518,6 +519,7 @@ The server supports all four OPC UA HistoryRead variants:
|
||||
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`, `StandardDeviation` (the CLI's own list; `SqlServer` serves thirteen aggregates — see the [aggregate table](#aggregates-processed-reads)) |
|
||||
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
|
||||
| **Events** | Equipment folder (event notifier) | n/a |
|
||||
| **Modified** (Raw with `IsReadModified = true`, #668) | Historized variable | `--modified` (see [Modified-value history](#modified-value-history-part-11)) |
|
||||
|
||||
**Variable nodes** (historized tags) serve Raw, Processed, and AtTime history.
|
||||
`Historizing=true` and `AccessLevels.HistoryRead` are set at materialization so any compliant
|
||||
@@ -635,6 +637,48 @@ says is complete carries no continuation point. `NumValuesPerNode == 0` stays un
|
||||
- Reverse event reads (`start > end`) are not paged server-side; no continuation point is issued for them.
|
||||
- No session on the request (in-process callers) degrades to single-shot, as for Raw.
|
||||
|
||||
### Modified-value history (Part 11)
|
||||
|
||||
**Gitea #668.** On the `SqlServer` and `QuestDb` backends every HistoryUpdate leaves a **modification trail**: one row per sample it
|
||||
inserted, replaced or deleted, carrying the value **as it was before** the change (none for an insert), the caller's principal, the
|
||||
modification time and the Part 11 `HistoryUpdateType`. A HistoryReadRaw with `IsReadModified = true` returns that trail as a
|
||||
`HistoryModifiedData` — one `DataValue` (the prior value) plus one parallel `ModificationInfo` (`ModificationTime`, `UpdateType`,
|
||||
`UserName`) per entry.
|
||||
|
||||
| Operation | Recorded `UpdateType` | Prior value |
|
||||
|---|---|---|
|
||||
| Insert | `Insert` | none — served `null` / `BadNoData` |
|
||||
| Replace | `Replace` | the replaced value and its status |
|
||||
| Update (upsert) | `Update` | the replaced value, or none when it inserted |
|
||||
| DeleteAtTime / DeleteRaw | `Delete` | the deleted value and its status |
|
||||
|
||||
- **Only applied items are recorded.** A refused item (`BadEntryExists`, `BadNoEntryExists`, `BadInvalidTimestamp`, …), a DeleteRaw over an
|
||||
empty range, the **live recorder's writes** and the **retention purge** leave nothing.
|
||||
- **`UserName` is the session's principal** — the API key's `NodeAcl` principal, or `anonymous` for an admitted anonymous session (the
|
||||
key id is stored too, not served). Longer than 200 characters is truncated, never refused.
|
||||
- **Window and order.** Same windows as Raw: forward `[start, end)` ascending by `(SourceTime, ModifiedId)`, reverse (`start > end`)
|
||||
`(end, start]` descending, `start == end` the modifications at exactly that time; an unset bound with a cap pages open-ended like Raw.
|
||||
Several modifications of one timestamp appear in the order they were made.
|
||||
- **Paging.** `NumValuesPerNode > 0` pages with session-bound continuation points keyed on the last entry's `(SourceTime, ModifiedId)` —
|
||||
an exact, exclusive key, so there are no ties to over-fetch and no trailing empty page (the backend reads `cap + 1` rows to know).
|
||||
The cursor is bound to the node and to the modified read kind (a raw or events point is `BadContinuationPointInvalid`).
|
||||
`NumValuesPerNode == 0` is unbounded.
|
||||
- **Authorization** is the ordinary **HistoryRead** decision — no new operation or permission (ungated with `Security:Acl:Enabled` off).
|
||||
- **Unsupported** ⇒ `BadHistoryOperationUnsupported`: the `Gateway` backend (the AVEVA Historian exposes no modification trail) and the
|
||||
`Null` default. **Deleting the trail** (`DeleteRawModifiedDetails.IsDeleteModified = true`) is not supported either — the trail is an
|
||||
audit record and is aged out only by retention (below).
|
||||
- **Atomicity differs by backend.** `SqlServer` writes the trail row with `OUTPUT … INTO` in the **same statement** that changes `hist.Sample`,
|
||||
so a change and its trail row commit or roll back together. QuestDB has no multi-table transaction, so the trail row is written **before**
|
||||
the change: a crash (or failed sample INSERT) in between leaves an **orphan trail row** describing a change that never landed — never a
|
||||
change without its trail row. Treat a QuestDB trail entry whose sample still shows the prior value as such an orphan.
|
||||
- **Retention.** The trail ages out on the same **source-time** cutoff as the samples it describes: `SqlServer` purges `hist.SampleModified`
|
||||
per tag alongside `hist.Sample`; QuestDB puts the same TTL on `{p}sample_modified`.
|
||||
|
||||
```bash
|
||||
otopcua-cli historyread -u opc.tcp://localhost:4840 -U apikey -P otua_<keyId>_<secret> \
|
||||
-n "ns=2;s=Plant/Modbus/dev1/Speed" --start 2026-01-15T00:00:00Z --end 2026-01-16T00:00:00Z --modified
|
||||
```
|
||||
|
||||
### Total aggregate
|
||||
|
||||
The OPC UA `Total` aggregate is **supported** over the HistorianGateway backend. The gateway exposes a
|
||||
@@ -665,13 +709,9 @@ member upstream. A NodeId outside the mapped set (e.g. `StandardDeviationSample`
|
||||
sample per requested timestamp, so the single-shot backend returns the complete result in one
|
||||
read and there is no "full page ⇒ maybe more" signal to page on. Returning the full result with
|
||||
no continuation point is spec-conformant.
|
||||
- **No modified-value history** (`HistoryReadModified`). Requests for modified values return
|
||||
`BadHistoryOperationUnsupported`. This is **infra-gated, not a server-code gap**: the HistorianGateway
|
||||
backend (`GatewayHistorianDataSource`) exposes only a current-value read path — there is no
|
||||
modified/edited-history surface to source the data from. The server-side override is in place (it cleanly
|
||||
rejects modified reads per node) and `IsReadModified` is honoured; serving real modified-value history is
|
||||
unblocked only once the gateway grows a modified-read RPC. Until then, rejecting is the correct,
|
||||
spec-conformant behaviour.
|
||||
- **No modified-value history on the `Gateway` backend.** Modified reads (`IsReadModified = true`) are served by
|
||||
`SqlServer` and `QuestDb` ([Modified-value history](#modified-value-history-part-11), #668); the HistorianGateway exposes no
|
||||
modified/edited-history surface, so on `Gateway` (and with no historian) they return `BadHistoryOperationUnsupported`.
|
||||
|
||||
### Redundancy and authorization
|
||||
|
||||
@@ -692,15 +732,18 @@ bootstrapper and one tag catalog, so the schema is ensured once per process and
|
||||
|
||||
### Schema
|
||||
|
||||
Created by the embedded idempotent script `Schema/001_initial.sql` (mirrored at
|
||||
[`docs/historian/sqlserver/001_initial.sql`](historian/sqlserver/001_initial.sql)); `$(Schema)` is the configured schema.
|
||||
Created by the embedded idempotent scripts `Schema/001_initial.sql` and `Schema/002_sample_modified.sql` (mirrored at
|
||||
[`docs/historian/sqlserver/001_initial.sql`](historian/sqlserver/001_initial.sql) and
|
||||
[`docs/historian/sqlserver/002_sample_modified.sql`](historian/sqlserver/002_sample_modified.sql)); `$(Schema)` is the configured schema.
|
||||
The bootstrapper applies every script above the recorded version, in order, under the schema applock.
|
||||
|
||||
| Object | Purpose |
|
||||
|---|---|
|
||||
| `SchemaVersion(Version, AppliedUtc)` | Applied-script ledger. Required version: **1**. |
|
||||
| `SchemaVersion(Version, AppliedUtc)` | Applied-script ledger. Required version: **2** (since #668). |
|
||||
| `Tag(TagId int identity, Name nvarchar(400) unique, DataType tinyint, EngineeringUnit, Description, CreatedUtc, UpdatedUtc)` | One row per historian tagname. `DataType` is a `DriverDataType`. |
|
||||
| `Sample(TagId, SourceTimeUtc datetime2(7), ValueKind tinyint, NumValue float, IntValue bigint, StrValue nvarchar(max), StatusCode int, ServerTimeUtc)` | **PK `(TagId, SourceTimeUtc)`** — one value per timestamp (Part 11), which makes redelivery idempotent and rules out tie clusters. `StatusCode` is the uint status stored bit-for-bit. |
|
||||
| `Event(EventId nvarchar(64) PK nonclustered, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, Message, UserName, Comment, EventTimeUtc, ReceivedTimeUtc)` | Alarm history. Clustered on `(EventTimeUtc, EventId)`; extra index `(SourceName, EventTimeUtc)`. |
|
||||
| `SampleModified(TagId, SourceTimeUtc, ModifiedId bigint identity, ModificationTimeUtc, UpdateType tinyint, Principal nvarchar(200), ApiKeyId, ValueKind, NumValue, IntValue, StrValue, StatusCode, ServerTimeUtc)` (v2) | The Part 11 [modification trail](#modified-value-history-part-11): one row per sample a HistoryUpdate changed, carrying the **prior** value columns (all NULL for an insert). Clustered PK `(TagId, SourceTimeUtc, ModifiedId)`. ⚠️ No FOREIGN KEY, CHECK or trigger — it is an `OUTPUT … INTO` target, which forbids them; adding one fails every HistoryUpdate. |
|
||||
| `SampleRow`, `TimeRow` table types | TVPs for batched sample writes/merges, AtTime and delete-at-time. |
|
||||
|
||||
Column mapping per `ValueKind`: `Double` → `NumValue`; `Int64` → `IntValue` + `NumValue` (as a double, for aggregates);
|
||||
@@ -708,9 +751,15 @@ Column mapping per `ValueKind`: `Double` → `NumValue`; `Int64` → `IntValue`
|
||||
`ValueKind` codes: Empty 0, Double 1, Int64 2, Boolean 3, String 4, DateTime 5. Timestamps are bound as `datetime2(7)`
|
||||
everywhere (full 100 ns precision — never a `DataTable`, which would round to 3.33 ms).
|
||||
|
||||
**Applying the script by hand** (needed when `AutoCreateSchema=false`): create the database, then
|
||||
`sqlcmd -S <server> -U <user> -P <pw> -d <db> -I -v Schema=hist -i docs/historian/sqlserver/001_initial.sql`
|
||||
(`-v Schema=` supplies the `$(Schema)` placeholder; verified against the docker-dev SQL container). Re-running is a no-op.
|
||||
**Applying the scripts by hand** (needed when `AutoCreateSchema=false`): create the database, then run each script in order —
|
||||
`sqlcmd -S <server> -U <user> -P <pw> -d <db> -I -v Schema=hist -i docs/historian/sqlserver/001_initial.sql`, then the same with
|
||||
`002_sample_modified.sql` (`-v Schema=` supplies the `$(Schema)` placeholder; verified against the docker-dev SQL container). Re-running is a no-op.
|
||||
|
||||
⚠️ **Upgrading an existing v1 historian (#668).** A build with this change requires schema **version 2**. With `AutoCreateSchema=true`
|
||||
(the default) the first historian call after the upgrade applies `002_sample_modified.sql` in place (it only creates `SampleModified`; existing
|
||||
samples are untouched). With `AutoCreateSchema=false` **every historian call fails** (`missing version 2 (found 1)`, in health `LastError`)
|
||||
until an operator runs `002_sample_modified.sql` by hand as above — do that before rolling the new build out. Modifications made before the
|
||||
upgrade have no trail; the trail starts with the first HistoryUpdate after it.
|
||||
|
||||
### Writes
|
||||
|
||||
@@ -826,7 +875,7 @@ The read client reports `GetHealthSnapshot()` (query/failure counters, consecuti
|
||||
|
||||
### Retention (`RetentionDays`)
|
||||
|
||||
`hist.Sample` and `hist.Event` grow without bound unless `ServerHistorian:SqlServer:RetentionDays` is set. QuestDB already has an
|
||||
`hist.Sample`, `hist.SampleModified` and `hist.Event` grow without bound unless `ServerHistorian:SqlServer:RetentionDays` is set. QuestDB already has an
|
||||
equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the AVEVA Historian's own.
|
||||
|
||||
- **What runs.** `HistorianRetentionService` (a hosted service, driver-role nodes, Backend=`SqlServer` with `RetentionDays` set; inert
|
||||
@@ -838,15 +887,17 @@ equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the
|
||||
database** — if they differ, the smallest window wins.
|
||||
- **How it deletes.** Cutoff = `UtcNow - RetentionDays` on source time. Samples are purged **per tag** with
|
||||
`DELETE TOP (@n) ... WHERE TagId = @id AND SourceTimeUtc < @cutoff` (a seek on the clustered `(TagId, SourceTimeUtc)` key; a global
|
||||
time predicate would scan), then events with `DELETE TOP (@n) ... WHERE EventTimeUtc < @cutoff` (seek on `CX_Event_Time`), each looped
|
||||
time predicate would scan) — and, in the same per-tag pass, the modification trail with
|
||||
`DELETE TOP (@n) FROM SampleModified WHERE TagId = @id AND SourceTimeUtc < @cutoff` (#668; same **source-time** cutoff, clustered seek; the
|
||||
purge itself never writes a trail row) — then events with `DELETE TOP (@n) ... WHERE EventTimeUtc < @cutoff` (seek on `CX_Event_Time`), each looped
|
||||
until a batch comes back short. Cancellation is honoured between batches. The `hist.Tag` catalogue is never purged.
|
||||
- **Shared databases.** Several pairs may point at one database; a session `sp_getapplock` (`OtOpcUa.Historian.Retention.<schema>`,
|
||||
timeout 0) lets one purger run and the others skip that tick (Debug log).
|
||||
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|Event}`; a failed pass logs a Warning and retries next
|
||||
- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|SampleModified|Event}`; a failed pass logs a Warning and retries next
|
||||
interval — it never affects capture or reads.
|
||||
- **Not done: partition switching.** For very large installs (billions of rows) a partitioned `hist.Sample` with sliding-window
|
||||
`SWITCH`/drop is cheaper than row deletes; the purger is deliberately simple batched deletes. Another table (e.g. a future
|
||||
`hist.SampleModified`, #668) is added by one more `DeleteLoopAsync` call in `SqlServerHistorianRetention.PurgeAsync`.
|
||||
`SWITCH`/drop is cheaper than row deletes; the purger is deliberately simple batched deletes. Another table is added by one more
|
||||
`DeleteLoopAsync` call in `SqlServerHistorianRetention.PurgeAsync` (as `hist.SampleModified` was, #668).
|
||||
|
||||
### Docker-dev recipe
|
||||
|
||||
@@ -911,10 +962,10 @@ Read only when `Backend=QuestDb`; bound by `HistorianBackendRegistry.BindQuestDb
|
||||
| Key | Default | Validation | Meaning |
|
||||
|---|---|---|---|
|
||||
| `QuestDb:ConnectionString` | `""` | **Required** (non-blank) | Npgsql connection string for QuestDB's PG-wire port, e.g. `Host=questdb;Port=8812;Username=admin;Password=quest;Database=qdb;ServerCompatibilityMode=NoTypeLoading`. `ServerCompatibilityMode=NoTypeLoading` skips the `pg_type` load; every query form was verified with it. Supply via env `ServerHistorian__QuestDb__ConnectionString` / `${secret:}` for anything real. |
|
||||
| `QuestDb:TablePrefix` | `hist_` | `^[A-Za-z0-9_]+$` | QuestDB has no schemas, so the three tables are `{prefix}sample`, `{prefix}event`, `{prefix}tag`. Spliced into DDL/DML, hence the regex. |
|
||||
| `QuestDb:RetentionDays` | unset | `>= 1` when set | Applied as table TTL to `{prefix}sample` and `{prefix}event` (see [Retention](#retention-ttl)). Unset keeps data forever. |
|
||||
| `QuestDb:TablePrefix` | `hist_` | `^[A-Za-z0-9_]+$` | QuestDB has no schemas, so the four tables are `{prefix}sample`, `{prefix}event`, `{prefix}tag` and `{prefix}sample_modified` (#668). Spliced into DDL/DML, hence the regex. |
|
||||
| `QuestDb:RetentionDays` | unset | `>= 1` when set | Applied as table TTL to `{prefix}sample`, `{prefix}event` and `{prefix}sample_modified` (see [Retention](#retention-ttl)). Unset keeps data forever. |
|
||||
| `QuestDb:CommandTimeout` | `00:00:30` | `> 0` | Per-command timeout, rounded up to whole seconds (minimum 1 s). |
|
||||
| `QuestDb:AutoCreateSchema` | `true` | — | `true`: create the tables (and reconcile the TTL) on first use. `false`: only verify the three tables exist; if one is missing every historian call fails with `HistorianSchemaMissingException` (health `LastError`, no crash) and no TTL is touched. A failed bootstrap is not latched: it is re-thrown for 5 s without touching QuestDB, then retried. |
|
||||
| `QuestDb:AutoCreateSchema` | `true` | — | `true`: create the tables (and reconcile the TTL) on first use. `false`: only verify the four tables exist (⚠️ an install from before #668 lacks `{prefix}sample_modified` — create it with the DDL in `QuestDbTables.CreateSampleModified` before upgrading, or every call fails); if one is missing every historian call fails with `HistorianSchemaMissingException` (health `LastError`, no crash) and no TTL is touched. A failed bootstrap is not latched: it is re-thrown for 5 s without touching QuestDB, then retried. |
|
||||
| `QuestDb:VisibilityTimeout` | `00:00:05` | `>= 0` | How long a write waits for its rows to become visible to queries ([WAL visibility](#wal-visibility-and-the-read-your-writes-wait)). `0` disables the wait. |
|
||||
|
||||
⚠️ **Connection pool size.** QuestDB's default limit is **64** concurrent PG-wire connections (`pg.net.connection.limit`), while Npgsql's
|
||||
@@ -926,14 +977,15 @@ limit is server-configurable. The effective value is logged once at Information
|
||||
|
||||
### Tables
|
||||
|
||||
All three are **WAL tables with `DEDUP UPSERT KEYS`**, so re-sending a row with the same key replaces it (the upsert is what makes
|
||||
HistoryUpdate Replace work; the writer's own insert-if-absent semantics are built on top, below).
|
||||
All are **WAL tables**; the first three carry **`DEDUP UPSERT KEYS`**, so re-sending a row with the same key replaces it (the upsert is what
|
||||
makes HistoryUpdate Replace work; the writer's own insert-if-absent semantics are built on top, below). The modification trail is append-only.
|
||||
|
||||
| Table | Key | Partitioning |
|
||||
|---|---|---|
|
||||
| `{p}sample` (`ts, tag, kind, num_value, int_value, str_value, status, deleted, server_ts`) | `(ts, tag)` | `PARTITION BY DAY` (TTL when configured) |
|
||||
| `{p}event` (one-for-one with `hist.Event`) | `(ts, event_id)` | `PARTITION BY DAY` (TTL when configured) |
|
||||
| `{p}tag` (`ts, name, data_type, eng_unit, description, updated_utc`) | `(ts, name)` with `ts` fixed at `1970-01-01` — an upsert by name | `PARTITION BY YEAR`, never expires |
|
||||
| `{p}sample_modified` (`ts` = the sample's source time, `tag, modified_id, modification_ts, update_type, principal, api_key_id, kind, num_value, int_value, str_value, status, server_ts`) (#668) | none — append-only, no DEDUP; `modified_id` (process-monotonic ≈ UTC ticks) orders a timestamp's trail | `PARTITION BY DAY` (TTL when configured). A NULL `status` marks "no prior value" (QuestDB `BYTE` has no NULL, so `kind` is 0) |
|
||||
|
||||
`status` holds the uint OPC UA status code **unsigned** in a `LONG` (so "Bad" is `status >= 2147483648`, not `< 0`); `int_value` also holds
|
||||
Booleans as 0/1; `kind` is the `HistorianValueKind` in a `BYTE`. Tag names are limited to 400 characters (longer is a per-tag failure,
|
||||
@@ -1000,6 +1052,11 @@ present **and not deleted**.
|
||||
| DeleteAtTime | write a tombstone (`deleted = true`, value NULL, `BadNoData`) where a live row exists | `Good` / `BadNoEntryExists` |
|
||||
| DeleteRaw | tombstone every live row in `[start, end)`, 1,000 per page, waiting for visibility between pages | the same codes as `SqlServer` |
|
||||
|
||||
**Modification trail (#668).** Each operation first appends one `{p}sample_modified` row per sample it will change, carrying the live row's
|
||||
current value (read by the same existence query), then writes the change. No multi-table transaction exists, so this order is the guarantee:
|
||||
a crash between the two leaves an **orphan trail row**, never an untracked change — the documented deviation from `SqlServer`'s atomic trail.
|
||||
See [Modified-value history](#modified-value-history-part-11).
|
||||
|
||||
The per-item codes come from the shared `HistoryUpdatePlanner`, so they equal `SqlServer`'s (including the duplicate-timestamp-in-one-request rules
|
||||
above). **Every read and aggregate filters `NOT deleted`.** A client Insert at a deleted timestamp re-populates it (the OPC UA intent); only the
|
||||
*live writer's* redelivery is barred from doing so.
|
||||
@@ -1025,7 +1082,7 @@ Every query filters `NOT deleted`; every value goes through `StoredSample`, so t
|
||||
|
||||
### Retention (TTL)
|
||||
|
||||
`RetentionDays` becomes `TTL n DAYS` on `{p}sample` and `{p}event` (never on `{p}tag`: TTL must be a multiple of the partition size). QuestDB drops
|
||||
`RetentionDays` becomes `TTL n DAYS` on `{p}sample`, `{p}event` and `{p}sample_modified` (never on `{p}tag`: TTL must be a multiple of the partition size). QuestDB drops
|
||||
expired day partitions itself, tombstones included.
|
||||
|
||||
- At bootstrap the current TTL is read from `tables()` (`ttlValue`, `ttlUnit`). QuestDB **normalises the unit** (7 days reads `1 WEEK`), so the
|
||||
@@ -1089,7 +1146,8 @@ batch limiter and `HistoryReadDeadline`.
|
||||
| `DeleteAtTimeDetails` | `HistoryDeleteAtTime` | per time: `Good`; `BadNoEntryExists` |
|
||||
|
||||
Each backend call is applied as a unit (SqlServer: one transaction, `MERGE … OUTPUT $action` against the TVP; QuestDb: an INSERT per
|
||||
chunk plus the WAL-visibility wait, no transaction — see [QuestDB backend](#questdb-backend)). The node-level `StatusCode` stays `Good` when item results are
|
||||
chunk plus the WAL-visibility wait, no transaction — see [QuestDB backend](#questdb-backend)). Every applied item is also recorded, with the
|
||||
caller's principal, in the [modification trail](#modified-value-history-part-11) (#668). The node-level `StatusCode` stays `Good` when item results are
|
||||
mixed (Part 11 reports item outcomes in `OperationResults`); it is Bad only for whole-node failures.
|
||||
|
||||
**Per-item rejections** (answered without touching SQL; other items still proceed): a null item ⇒ `BadInvalidArgument`;
|
||||
@@ -1113,8 +1171,8 @@ outcome, later ones `BadNoEntryExists`. **Unknown tag:** Insert / Update create
|
||||
(DeleteRaw: `BadNoData`) without creating anything.
|
||||
|
||||
**Unsupported ⇒ `BadHistoryOperationUnsupported`** (per node): the node is not historized; the backend has no updater (`Gateway`, or
|
||||
`ServerHistorian:Enabled=false`); `PerformUpdateType.Remove`; `DeleteRawModifiedDetails.IsDeleteModified=true` (no modified-value history is
|
||||
kept); and the out-of-scope detail types — **event** HistoryUpdate (`UpdateEventDetails` / `DeleteEventDetails`) and `UpdateStructureData` —
|
||||
`ServerHistorian:Enabled=false`); `PerformUpdateType.Remove`; `DeleteRawModifiedDetails.IsDeleteModified=true` (the
|
||||
[modification trail](#modified-value-history-part-11) is an audit record, aged out only by retention); and the out-of-scope detail types — **event** HistoryUpdate (`UpdateEventDetails` / `DeleteEventDetails`) and `UpdateStructureData` —
|
||||
which are not overridden, so the SDK base rejects them. A backend exception is `BadInternalError` for that node and a deadline is `BadTimeout`.
|
||||
No `AuditHistoryUpdateEvent` is emitted.
|
||||
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
-- OtOpcUa SQL Server historian backend - schema version 2: OPC UA Part 11 modified-value history (Gitea #668).
|
||||
-- $(Schema) is substituted by SqlServerSchemaBootstrapper (identifier-validated) BEFORE the script is split on GO lines.
|
||||
-- Every statement is guarded so a re-run is a no-op.
|
||||
--
|
||||
-- One row per sample a HistoryUpdate inserted, replaced or deleted, carrying the PRIOR value (all NULL for an
|
||||
-- insert). Rows are written by the SAME statement that changes hist.Sample (OUTPUT ... INTO), so a failed update
|
||||
-- leaves no audit row and a committed one always has its row.
|
||||
-- WARNING: OUTPUT ... INTO forbids, on its target table: enabled triggers, either side of a FOREIGN KEY, and CHECK
|
||||
-- constraints. Do not add any of those to SampleModified, or every HistoryUpdate fails.
|
||||
IF OBJECT_ID(N'[$(Schema)].[SampleModified]', N'U') IS NULL
|
||||
CREATE TABLE [$(Schema)].[SampleModified] (
|
||||
TagId int NOT NULL,
|
||||
SourceTimeUtc datetime2(7) NOT NULL,
|
||||
ModifiedId bigint IDENTITY(1,1) NOT NULL,
|
||||
ModificationTimeUtc datetime2(7) NOT NULL,
|
||||
UpdateType tinyint NOT NULL,
|
||||
Principal nvarchar(200) NOT NULL,
|
||||
ApiKeyId nvarchar(200) NULL,
|
||||
ValueKind tinyint NULL,
|
||||
NumValue float NULL,
|
||||
IntValue bigint NULL,
|
||||
StrValue nvarchar(max) NULL,
|
||||
StatusCode int NULL,
|
||||
ServerTimeUtc datetime2(7) NULL,
|
||||
CONSTRAINT PK_SampleModified PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc, ModifiedId))
|
||||
GO
|
||||
IF NOT EXISTS (SELECT 1 FROM [$(Schema)].[SchemaVersion] WHERE Version = 2)
|
||||
INSERT [$(Schema)].[SchemaVersion] (Version, AppliedUtc) VALUES (2, SYSUTCDATETIME())
|
||||
GO
|
||||
@@ -884,6 +884,12 @@ HistoryUpdate rewrites recorded history, so unlike HistoryRead it is gated **wha
|
||||
- **Flag off (default):** the session's API key is re-checked on the call (`DataPlaneAuthorizer.AuthorizeKeyUsable`, Gitea #563 — an expired or disabled key loses its activation-time roles), then the session must carry the **`HistoryUpdate` role** (`OpcUaDataPlaneRoles.HistoryUpdate`). `ApiKeyInterimRoleMapper` confers it **only from a `Cluster`-scoped grant** carrying `NodePermissions.HistoryUpdate` — narrower-scoped grants confer nothing, for the same reason as `WriteOperate` (honouring a flag without honouring its scope would widen it). A key holding only `WriteOperate` is denied `BadUserAccessDenied`.
|
||||
- **Flag on:** `DataPlaneAuthorizer.Authorize(identity, OpcUaOperation.HistoryUpdate, nodeId, realm)` decides per node and **replaces** the role check (never chained) — a session with no roles succeeds on the strength of a scoped `HistoryUpdate` grant, and a `WriteOperate` grant does not help.
|
||||
- **Availability is separate from authorization.** A node accepts HistoryUpdate only if it is historized **and** the selected historian backend has an updater when the node materialises (the `HistoryWrite` AccessLevel bit). `Backend=Gateway` has none, so it answers `BadHistoryOperationUnsupported` for every node whatever the grant. Operations, result codes and unsupported cases: [`docs/Historian.md` §HistoryUpdate](Historian.md#historyupdate).
|
||||
- **Every applied change is attributed (Part 11 modified-value history, Gitea #668).** The session's principal (the API key's
|
||||
`NodeAcl` principal, or the reserved `anonymous`) and key id are passed to the backend with every HistoryUpdate and recorded in the
|
||||
modification trail next to the prior value; a modified-value HistoryRead (`IsReadModified = true`) returns it as
|
||||
`ModificationInfo.UserName`. Reading the trail is gated by the **same `HistoryRead` decision** as a raw read — there is no separate
|
||||
permission — so with `Security:Acl:Enabled` off it is as ungated as any HistoryRead, and anyone who may read a node's history can see
|
||||
who rewrote it and what it held before. See [`docs/Historian.md` §Modified-value history](Historian.md#modified-value-history-part-11).
|
||||
|
||||
### Evaluator — `PermissionTrie`
|
||||
|
||||
|
||||
@@ -56,6 +56,13 @@ public class HistoryReadCommand : CommandBase
|
||||
[CommandOption("interval", Description = "Processing interval in milliseconds for aggregates")]
|
||||
public double IntervalMs { get; init; } = 3600000;
|
||||
|
||||
/// <summary>
|
||||
/// Gets a value indicating whether to read modified-value history (Part 11 IsReadModified) instead of raw samples:
|
||||
/// the prior value each HistoryUpdate replaced or deleted, with its update type, user and modification time.
|
||||
/// </summary>
|
||||
[CommandOption("modified", Description = "Read modified-value history (what HistoryUpdate changed: prior value, update type, user, time) instead of raw samples")]
|
||||
public bool Modified { get; init; }
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async ValueTask ExecuteAsync(IConsole console)
|
||||
{
|
||||
@@ -64,6 +71,8 @@ public class HistoryReadCommand : CommandBase
|
||||
throw new CommandException($"--max must be greater than 0 (was {MaxValues}).");
|
||||
if (!string.IsNullOrEmpty(Aggregate) && IntervalMs <= 0)
|
||||
throw new CommandException($"--interval must be greater than 0 (was {IntervalMs}).");
|
||||
if (Modified && !string.IsNullOrEmpty(Aggregate))
|
||||
throw new CommandException("--modified and --aggregate cannot be combined.");
|
||||
|
||||
NodeId nodeId;
|
||||
try
|
||||
@@ -119,6 +128,30 @@ public class HistoryReadCommand : CommandBase
|
||||
var ct = console.RegisterCancellationHandler();
|
||||
(service, _) = await CreateServiceAndConnectAsync(ct);
|
||||
|
||||
if (Modified)
|
||||
{
|
||||
await console.Output.WriteLineAsync(
|
||||
$"Modified history for {NodeId} ({start:yyyy-MM-dd HH:mm} -> {end:yyyy-MM-dd HH:mm})");
|
||||
var entries = await service.HistoryReadModifiedAsync(nodeId, start, end, MaxValues, ct);
|
||||
await console.Output.WriteLineAsync();
|
||||
await console.Output.WriteLineAsync(
|
||||
$"{"Timestamp",-35} {"Prior value",-15} {"Status",-10} {"Update",-8} {"User",-20} {"Modified at"}");
|
||||
foreach (var e in entries)
|
||||
{
|
||||
var dv = e.Value;
|
||||
var status = StatusCode.IsGood(dv.StatusCode) ? "Good"
|
||||
: StatusCode.IsBad(dv.StatusCode) ? "Bad"
|
||||
: "Uncertain";
|
||||
await console.Output.WriteLineAsync(
|
||||
$"{dv.SourceTimestamp.ToString("O"),-35} {dv.Value?.ToString() ?? "(none)",-15} {status,-10} " +
|
||||
$"{e.UpdateType,-8} {e.UserName ?? "",-20} {e.ModificationTime:O}");
|
||||
}
|
||||
|
||||
await console.Output.WriteLineAsync();
|
||||
await console.Output.WriteLineAsync($"{entries.Count} modifications returned.");
|
||||
return;
|
||||
}
|
||||
|
||||
IReadOnlyList<DataValue> values;
|
||||
|
||||
if (string.IsNullOrEmpty(Aggregate))
|
||||
|
||||
@@ -206,6 +206,74 @@ internal sealed class DefaultSessionAdapter : ISessionAdapter
|
||||
return allValues;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<IReadOnlyList<Models.HistoryModifiedEntry>> HistoryReadModifiedAsync(
|
||||
NodeId nodeId, DateTime startTime, DateTime endTime, int maxValues, CancellationToken ct)
|
||||
{
|
||||
var details = new ReadRawModifiedDetails
|
||||
{
|
||||
StartTime = startTime,
|
||||
EndTime = endTime,
|
||||
NumValuesPerNode = (uint)maxValues,
|
||||
IsReadModified = true,
|
||||
ReturnBounds = false
|
||||
};
|
||||
|
||||
var nodesToRead = new HistoryReadValueIdCollection
|
||||
{
|
||||
new HistoryReadValueId { NodeId = nodeId }
|
||||
};
|
||||
|
||||
var all = new List<Models.HistoryModifiedEntry>();
|
||||
byte[]? continuationPoint = null;
|
||||
|
||||
do
|
||||
{
|
||||
if (continuationPoint != null)
|
||||
nodesToRead[0].ContinuationPoint = continuationPoint;
|
||||
|
||||
var response = await _session.HistoryReadAsync(
|
||||
null,
|
||||
new ExtensionObject(details),
|
||||
TimestampsToReturn.Both,
|
||||
false,
|
||||
nodesToRead,
|
||||
ct).ConfigureAwait(false);
|
||||
|
||||
var results = response.Results;
|
||||
if (results == null || results.Count == 0)
|
||||
break;
|
||||
|
||||
var result = results[0];
|
||||
HistoryReadStatus.ThrowIfRefused(result.StatusCode, nodeId);
|
||||
|
||||
if (result.HistoryData is ExtensionObject ext && ext.Body is HistoryModifiedData modified)
|
||||
{
|
||||
for (var i = 0; i < modified.DataValues.Count; i++)
|
||||
{
|
||||
var info = i < modified.ModificationInfos.Count ? modified.ModificationInfos[i] : null;
|
||||
all.Add(new Models.HistoryModifiedEntry(
|
||||
modified.DataValues[i],
|
||||
info?.ModificationTime ?? DateTime.MinValue,
|
||||
info?.UpdateType ?? default,
|
||||
info?.UserName));
|
||||
}
|
||||
}
|
||||
|
||||
continuationPoint = result.ContinuationPoint;
|
||||
} while (continuationPoint != null && continuationPoint.Length > 0 && all.Count < maxValues);
|
||||
|
||||
// A read stopped by the client's own cap leaves a server cursor behind; release it.
|
||||
if (continuationPoint is { Length: > 0 })
|
||||
{
|
||||
nodesToRead[0].ContinuationPoint = continuationPoint;
|
||||
await _session.HistoryReadAsync(null, new ExtensionObject(details), TimestampsToReturn.Both, true,
|
||||
nodesToRead, ct).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<IReadOnlyList<DataValue>> HistoryReadAggregateAsync(
|
||||
NodeId nodeId, DateTime startTime, DateTime endTime, NodeId aggregateId, double intervalMs,
|
||||
|
||||
@@ -110,6 +110,20 @@ internal interface ISessionAdapter : IDisposable
|
||||
Task<IReadOnlyList<DataValue>> HistoryReadRawAsync(NodeId nodeId, DateTime startTime, DateTime endTime,
|
||||
int maxValues, CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Reads modified-value history (Part 11, <c>IsReadModified = true</c>): the prior values and modification info
|
||||
/// the server recorded for every HistoryUpdate in the window, following continuation points.
|
||||
/// </summary>
|
||||
/// <param name="nodeId">The historized node.</param>
|
||||
/// <param name="startTime">Window start.</param>
|
||||
/// <param name="endTime">Window end.</param>
|
||||
/// <param name="maxValues">The maximum number of entries to return.</param>
|
||||
/// <param name="ct">The cancellation token.</param>
|
||||
/// <returns>The modification entries in server order.</returns>
|
||||
Task<IReadOnlyList<Models.HistoryModifiedEntry>> HistoryReadModifiedAsync(NodeId nodeId, DateTime startTime,
|
||||
DateTime endTime, int maxValues, CancellationToken ct = default) =>
|
||||
throw new NotSupportedException("This session adapter does not support modified-value history reads.");
|
||||
|
||||
/// <summary>
|
||||
/// Sends a HistoryUpdate request for the supplied details.
|
||||
/// </summary>
|
||||
|
||||
@@ -193,6 +193,20 @@ public interface IOpcUaClientService : IDisposable
|
||||
Task<IReadOnlyList<DataValue>> HistoryReadRawAsync(NodeId nodeId, DateTime startTime, DateTime endTime,
|
||||
int maxValues = 1000, CancellationToken ct = default);
|
||||
|
||||
/// <summary>
|
||||
/// Reads modified-value history (Part 11, <c>IsReadModified = true</c>): what each HistoryUpdate in the window
|
||||
/// replaced or deleted, with the server's modification time, update type and user name.
|
||||
/// </summary>
|
||||
/// <param name="nodeId">The historized node.</param>
|
||||
/// <param name="startTime">Window start.</param>
|
||||
/// <param name="endTime">Window end.</param>
|
||||
/// <param name="maxValues">The maximum number of entries to return.</param>
|
||||
/// <param name="ct">The cancellation token.</param>
|
||||
/// <returns>The modification entries in server order.</returns>
|
||||
Task<IReadOnlyList<HistoryModifiedEntry>> HistoryReadModifiedAsync(NodeId nodeId, DateTime startTime,
|
||||
DateTime endTime, int maxValues = 1000, CancellationToken ct = default) =>
|
||||
throw new NotSupportedException("This client service does not support modified-value history reads.");
|
||||
|
||||
/// <summary>
|
||||
/// Sends a HistoryUpdate request (insert/replace/update or delete) for the supplied details.
|
||||
/// </summary>
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
using Opc.Ua;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Client.Shared.Models;
|
||||
|
||||
/// <summary>
|
||||
/// One entry of an OPC UA Part 11 modified-value history read (HistoryReadRaw with <c>IsReadModified = true</c>): the
|
||||
/// value as it was BEFORE the modification plus the server's <see cref="ModificationInfo"/>.
|
||||
/// </summary>
|
||||
/// <param name="Value">The prior value at its source timestamp (an insert had none: null value, Bad status).</param>
|
||||
/// <param name="ModificationTime">When the modification was applied.</param>
|
||||
/// <param name="UpdateType">The Part 11 update type (Insert / Replace / Update / Delete).</param>
|
||||
/// <param name="UserName">Who made the modification (the server's recorded principal).</param>
|
||||
public sealed record HistoryModifiedEntry(
|
||||
DataValue Value,
|
||||
DateTime ModificationTime,
|
||||
HistoryUpdateType UpdateType,
|
||||
string? UserName);
|
||||
@@ -559,6 +559,15 @@ public sealed class OpcUaClientService : IOpcUaClientService
|
||||
return await _session!.HistoryReadRawAsync(nodeId, startTime, endTime, maxValues, ct);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<IReadOnlyList<HistoryModifiedEntry>> HistoryReadModifiedAsync(
|
||||
NodeId nodeId, DateTime startTime, DateTime endTime, int maxValues = 1000, CancellationToken ct = default)
|
||||
{
|
||||
ThrowIfDisposed();
|
||||
ThrowIfNotConnected();
|
||||
return await _session!.HistoryReadModifiedAsync(nodeId, startTime, endTime, maxValues, ct);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<HistoryUpdateOutcome> HistoryUpdateAsync(HistoryUpdateDetails details, CancellationToken ct = default)
|
||||
{
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
// OPC UA Part 11 modified-value history (Gitea #668): every HistoryUpdate a backend applies leaves an audit row
|
||||
// carrying the PRIOR value, who made the change and when; HistoryRead with IsReadModified=true serves them.
|
||||
|
||||
/// <summary>Who is performing a HistoryUpdate. Threaded into every <see cref="IHistorianDataUpdater"/> call.</summary>
|
||||
/// <param name="Principal">
|
||||
/// The session's authenticated principal (an API key's principal, or the reserved <c>anonymous</c>). Recorded as
|
||||
/// the modification's user name.
|
||||
/// </param>
|
||||
/// <param name="ApiKeyId">The API key id the session authenticated with, or <c>null</c> (anonymous / non-key).</param>
|
||||
public sealed record HistoryUpdateContext(string Principal, string? ApiKeyId)
|
||||
{
|
||||
/// <summary>The longest principal / key id a backend stores; longer values are truncated (never refused).</summary>
|
||||
public const int MaxIdentityLength = 200;
|
||||
|
||||
/// <summary>The principal clamped to <see cref="MaxIdentityLength"/> characters.</summary>
|
||||
public string StoredPrincipal => Clamp(Principal) ?? string.Empty;
|
||||
|
||||
/// <summary>The key id clamped to <see cref="MaxIdentityLength"/> characters.</summary>
|
||||
public string? StoredApiKeyId => Clamp(ApiKeyId);
|
||||
|
||||
private static string? Clamp(string? s) => s is { Length: > MaxIdentityLength } ? s[..MaxIdentityLength] : s;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The OPC UA Part 11 <c>HistoryUpdateType</c> of a recorded modification. The numeric values are the spec's and
|
||||
/// are persisted (SQL <c>tinyint</c>, QuestDB <c>BYTE</c>) — never renumber.
|
||||
/// </summary>
|
||||
public enum HistoryModificationType : byte
|
||||
{
|
||||
/// <summary>A sample was inserted where none existed (no prior value).</summary>
|
||||
Insert = 1,
|
||||
|
||||
/// <summary>An existing sample was replaced.</summary>
|
||||
Replace = 2,
|
||||
|
||||
/// <summary>An upsert (<c>PerformUpdateType.Update</c>): replaced a sample, or inserted one (then no prior value).</summary>
|
||||
Update = 3,
|
||||
|
||||
/// <summary>A sample was deleted (DeleteAtTime / DeleteRaw).</summary>
|
||||
Delete = 4,
|
||||
}
|
||||
|
||||
/// <summary>One recorded modification: the value as it was BEFORE the change, plus who changed it and when.</summary>
|
||||
/// <param name="Value">
|
||||
/// The prior value at its source timestamp. An insert had no prior value: <c>Value = null</c>,
|
||||
/// <c>StatusCode = BadNoData</c>.
|
||||
/// </param>
|
||||
/// <param name="ModificationTimeUtc">When the modification was applied.</param>
|
||||
/// <param name="UpdateType">The kind of modification.</param>
|
||||
/// <param name="UserName">The principal that made it.</param>
|
||||
/// <param name="ModifiedId">The backend's ordering key within one source timestamp (paging cursor).</param>
|
||||
public sealed record HistoryModifiedValue(
|
||||
DataValueSnapshot Value,
|
||||
DateTime ModificationTimeUtc,
|
||||
HistoryModificationType UpdateType,
|
||||
string UserName,
|
||||
long ModifiedId);
|
||||
|
||||
/// <summary>Resume position for a paged modified-value read: strictly after (or before, reversed) this key.</summary>
|
||||
/// <param name="SourceTimeUtc">The last returned modification's source time.</param>
|
||||
/// <param name="ModifiedId">The last returned modification's id.</param>
|
||||
public sealed record HistoryModifiedCursor(DateTime SourceTimeUtc, long ModifiedId);
|
||||
|
||||
/// <summary>Result of <see cref="IHistorianDataSource.ReadModifiedAsync"/>.</summary>
|
||||
/// <param name="Values">Modifications ordered by <c>(SourceTimeUtc, ModifiedId)</c> (descending for a reverse read).</param>
|
||||
/// <param name="HasMore"><c>true</c> when the cap truncated the result and more modifications remain in the window.</param>
|
||||
public sealed record HistoryModifiedReadResult(IReadOnlyList<HistoryModifiedValue> Values, bool HasMore);
|
||||
@@ -122,4 +122,32 @@ public interface IHistorianDataSource : IDisposable
|
||||
/// Same-timestamp ties are unaffected (they are paged within the timestamp, never stepped over).
|
||||
/// </remarks>
|
||||
long TimestampResolutionTicks => 1;
|
||||
|
||||
/// <summary>
|
||||
/// Read modified-value history — OPC UA HistoryReadRaw with <c>IsReadModified = true</c> (Part 11, Gitea #668):
|
||||
/// every modification a HistoryUpdate applied to the tag inside the window, each carrying the PRIOR value.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Window: forward (<c>start < end</c>) <c>[start, end)</c> ascending by <c>(SourceTimeUtc, ModifiedId)</c>;
|
||||
/// reverse (<c>end < start</c>) <c>(end, start]</c> descending; <c>start == end</c> the modifications at
|
||||
/// exactly that time. <paramref name="resumeAfter"/> continues strictly past that key in the read's direction.
|
||||
/// The default throws <see cref="NotSupportedException"/> — a backend that keeps no modification trail
|
||||
/// (Gateway, Null) — which the server reports as <c>BadHistoryOperationUnsupported</c>.
|
||||
/// </remarks>
|
||||
/// <param name="fullReference">The historian tagname.</param>
|
||||
/// <param name="startUtc">Window start (UTC).</param>
|
||||
/// <param name="endUtc">Window end (UTC).</param>
|
||||
/// <param name="maxValues">Cap on returned modifications; <c>0</c> is unbounded.</param>
|
||||
/// <param name="resumeAfter">Resume cursor from a previous page, or <c>null</c> for the first page.</param>
|
||||
/// <param name="cancellationToken">Cancellation.</param>
|
||||
/// <returns>The modifications, with <see cref="HistoryModifiedReadResult.HasMore"/> set when truncated.</returns>
|
||||
/// <exception cref="NotSupportedException">The backend keeps no modified-value history.</exception>
|
||||
Task<HistoryModifiedReadResult> ReadModifiedAsync(
|
||||
string fullReference,
|
||||
DateTime startUtc,
|
||||
DateTime endUtc,
|
||||
uint maxValues,
|
||||
HistoryModifiedCursor? resumeAfter,
|
||||
CancellationToken cancellationToken) =>
|
||||
throw new NotSupportedException("This historian backend keeps no modified-value history.");
|
||||
}
|
||||
|
||||
@@ -17,12 +17,20 @@ public enum HistoryUpdateMode
|
||||
/// Optional historian capability behind OPC UA HistoryUpdate. A backend that cannot modify history
|
||||
/// supplies no updater (the server then reports HistoryUpdate as unsupported).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <b>Modified-value history (Part 11, Gitea #668).</b> Every sample an implementation inserts, replaces or deletes
|
||||
/// MUST leave one modification record carrying the prior value (none for an insert), the
|
||||
/// <see cref="HistoryUpdateContext.Principal"/>, the modification time and the <see cref="HistoryModificationType"/>,
|
||||
/// readable through <see cref="IHistorianDataSource.ReadModifiedAsync"/>. A refused item (any non-Good code) leaves
|
||||
/// no record. Live recorder writes and retention purges never leave one.
|
||||
/// </remarks>
|
||||
public interface IHistorianDataUpdater
|
||||
{
|
||||
/// <summary>Inserts, replaces or upserts samples for one tag.</summary>
|
||||
/// <param name="tag">Historian tag name.</param>
|
||||
/// <param name="mode">Update mode.</param>
|
||||
/// <param name="values">Samples to apply.</param>
|
||||
/// <param name="context">Who is making the change (recorded in the modification trail).</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns>
|
||||
/// One status code per value, in order: Insert gives <c>GoodEntryInserted</c> or <c>BadEntryExists</c>;
|
||||
@@ -30,20 +38,24 @@ public interface IHistorianDataUpdater
|
||||
/// <c>GoodEntryInserted</c> or <c>GoodEntryReplaced</c>; a null timestamp gives <c>BadInvalidTimestamp</c>.
|
||||
/// </returns>
|
||||
Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
|
||||
IReadOnlyList<HistorizationValue> values, CancellationToken ct);
|
||||
IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context, CancellationToken ct);
|
||||
|
||||
/// <summary>Deletes raw samples in a time range.</summary>
|
||||
/// <param name="tag">Historian tag name.</param>
|
||||
/// <param name="startUtc">Range start (UTC).</param>
|
||||
/// <param name="endUtc">Range end (UTC).</param>
|
||||
/// <param name="context">Who is making the change (recorded in the modification trail).</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns><c>Good</c>, or <c>BadNoData</c> when the range held nothing.</returns>
|
||||
Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct);
|
||||
Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct);
|
||||
|
||||
/// <summary>Deletes the samples at exact timestamps.</summary>
|
||||
/// <param name="tag">Historian tag name.</param>
|
||||
/// <param name="timesUtc">Timestamps to delete (UTC).</param>
|
||||
/// <param name="context">Who is making the change (recorded in the modification trail).</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
/// <returns>One status code per time, in order: <c>Good</c> or <c>BadNoEntryExists</c>.</returns>
|
||||
Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct);
|
||||
Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct);
|
||||
}
|
||||
|
||||
@@ -83,6 +83,8 @@ internal sealed class QuestDbConnection : IAsyncDisposable, IDisposable
|
||||
public string EventTable => T(QuestDbTables.EventBase);
|
||||
/// <summary>The prefixed tag table.</summary>
|
||||
public string TagTable => T(QuestDbTables.TagBase);
|
||||
/// <summary>The prefixed Part 11 modification-trail table (#668).</summary>
|
||||
public string SampleModifiedTable => T(QuestDbTables.SampleModifiedBase);
|
||||
|
||||
/// <summary>Opens a pooled connection (no bootstrap; callers run the bootstrapper first).</summary>
|
||||
/// <param name="ct">Cancellation.</param>
|
||||
|
||||
@@ -137,6 +137,101 @@ public sealed class QuestDbHistorianDataSource : IHistorianDataSource
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------ Modified (Part 11, #668)
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
/// Reads <c>{prefix}sample_modified</c>, the append-only trail <see cref="QuestDbHistorianDataUpdater"/> writes
|
||||
/// BEFORE each change. Windows follow <see cref="ReadRawAsync"/> (bounds rounded to µs); rows are ordered by
|
||||
/// <c>(ts, modified_id)</c>; <c>maxValues + 1</c> rows are fetched to report
|
||||
/// <see cref="HistoryModifiedReadResult.HasMore"/> exactly. A row whose <c>status</c> is NULL had no prior value
|
||||
/// (an insert) and is served <c>null</c> / <c>BadNoData</c>. Because the trail row lands first, a crash between it and
|
||||
/// the sample write leaves an orphan row describing a change that never applied (documented).
|
||||
/// </remarks>
|
||||
public async Task<HistoryModifiedReadResult> ReadModifiedAsync(
|
||||
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValues, HistoryModifiedCursor? resumeAfter,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(fullReference);
|
||||
var bounded = maxValues > 0 && maxValues < int.MaxValue;
|
||||
try
|
||||
{
|
||||
var values = new List<HistoryModifiedValue>();
|
||||
var hasMore = false;
|
||||
if (await ResolveAsync(fullReference, cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
var start = QuestDbConnection.Round(startUtc);
|
||||
var end = QuestDbConnection.Round(endUtc);
|
||||
var reverse = start > end;
|
||||
string where;
|
||||
if (start == end) where = "ts = $2 AND ts = $3";
|
||||
else if (!reverse) where = "ts >= $2 AND ts < $3";
|
||||
else where = "ts > $2 AND ts <= $3";
|
||||
var next = 4;
|
||||
if (resumeAfter is not null)
|
||||
{
|
||||
where += reverse
|
||||
? " AND (ts < $4 OR (ts = $4 AND modified_id < $5))"
|
||||
: " AND (ts > $4 OR (ts = $4 AND modified_id > $5))";
|
||||
next = 6;
|
||||
}
|
||||
var order = reverse ? "DESC" : "ASC";
|
||||
var (lo, hi) = start <= end ? (start, end) : (end, start);
|
||||
|
||||
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
||||
await using var cmd = _conn.CreateCommand(conn,
|
||||
$"SELECT {SampleColumns}, modified_id, modification_ts, update_type, principal FROM {_conn.SampleModifiedTable} " +
|
||||
$"WHERE tag = $1 AND {where} ORDER BY ts {order}, modified_id {order}" + (bounded ? $" LIMIT ${next}" : ""));
|
||||
QuestDbConnection.BindText(cmd, fullReference);
|
||||
QuestDbConnection.BindTimestamp(cmd, lo);
|
||||
QuestDbConnection.BindTimestamp(cmd, hi);
|
||||
if (resumeAfter is not null)
|
||||
{
|
||||
QuestDbConnection.BindTimestamp(cmd, resumeAfter.SourceTimeUtc);
|
||||
QuestDbConnection.BindLong(cmd, resumeAfter.ModifiedId);
|
||||
}
|
||||
if (bounded)
|
||||
cmd.Parameters.Add(new NpgsqlParameter { NpgsqlDbType = NpgsqlDbType.Integer, Value = (int)maxValues + 1 });
|
||||
|
||||
await using var reader = await cmd.ExecuteReaderAsync(cancellationToken).ConfigureAwait(false);
|
||||
while (await reader.ReadAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
if (bounded && values.Count == maxValues)
|
||||
{
|
||||
hasMore = true;
|
||||
break;
|
||||
}
|
||||
|
||||
var sourceTime = QuestDbConnection.ReadUtc(reader, 0);
|
||||
var modifiedAt = QuestDbConnection.ReadUtc(reader, 8);
|
||||
DataValueSnapshot prior;
|
||||
if (reader.IsDBNull(5))
|
||||
{
|
||||
prior = new DataValueSnapshot(null, HistorianStatusCodes.BadNoData, sourceTime, modifiedAt);
|
||||
}
|
||||
else
|
||||
{
|
||||
var s = ReadStored(reader, 0);
|
||||
prior = new DataValueSnapshot(s.Value.ToObject(), s.Status, s.TimeUtc, ReadServerTime(reader, 6, modifiedAt));
|
||||
}
|
||||
|
||||
values.Add(new HistoryModifiedValue(prior, modifiedAt,
|
||||
(HistoryModificationType)QuestDbConnection.ReadKind(reader, 9),
|
||||
reader.IsDBNull(10) ? string.Empty : reader.GetString(10),
|
||||
reader.GetInt64(7)));
|
||||
}
|
||||
}
|
||||
|
||||
RecordSuccess();
|
||||
return new HistoryModifiedReadResult(values, hasMore);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
RecordFailure(ex);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------ AtTime
|
||||
|
||||
/// <inheritdoc />
|
||||
|
||||
+156
-14
@@ -25,6 +25,13 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb;
|
||||
/// 1970 (read bounds are not clamped).</para>
|
||||
/// <para><b>Read-your-writes:</b> after writing, <see cref="QuestDbWalVisibility"/> waits for the rows to become
|
||||
/// visible, bounded by <c>VisibilityTimeout</c>; a timeout only logs a Warning, never changes a code.</para>
|
||||
/// <para><b>Modified-value history (Part 11, #668).</b> Every sample this updater inserts, replaces or tombstones first
|
||||
/// gets a row in <c>{prefix}sample_modified</c> carrying its PRIOR live value (the existence read already fetches it;
|
||||
/// none for an insert — a tombstone is "absent", so an insert over one records no prior value), the caller's
|
||||
/// <see cref="HistoryUpdateContext"/>, the <see cref="HistoryModificationType"/> and the modification time. QuestDB has
|
||||
/// no multi-table transaction, so the trail row is written <b>BEFORE</b> the change: a crash (or a failed sample INSERT)
|
||||
/// between the two leaves an orphan trail row describing a change that never landed — never a change without its
|
||||
/// trail row. This is the documented deviation from SQL Server, whose trail commits atomically with the change.</para>
|
||||
/// </summary>
|
||||
internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
{
|
||||
@@ -35,6 +42,7 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
private const int LockStripes = 64;
|
||||
private readonly SemaphoreSlim[] _locks = Enumerable.Range(0, LockStripes).Select(_ => new SemaphoreSlim(1, 1)).ToArray();
|
||||
private long _liveRowsRead;
|
||||
private static long _lastModifiedId;
|
||||
private readonly QuestDbConnection _conn;
|
||||
private readonly QuestDbTagCatalog _tags;
|
||||
private readonly QuestDbWalVisibility _wal;
|
||||
@@ -71,10 +79,11 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
|
||||
IReadOnlyList<HistorizationValue> values, CancellationToken ct)
|
||||
IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(values);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
|
||||
// RC1: a valid-timestamp item before 1970 can never be stored; refuse it and plan the rest.
|
||||
var final = new uint[values.Count];
|
||||
@@ -100,7 +109,7 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
? await _tags.ExistsAsync(tag, ct).ConfigureAwait(false)
|
||||
: await EnsureAsync(tag, plan.Winners[0].Row.Value.Kind, ct).ConfigureAwait(false);
|
||||
var outcome = known
|
||||
? await ApplyAsync(tag, mode, plan.Winners.Select(w => w.Row).ToList(), ct).ConfigureAwait(false)
|
||||
? await ApplyAsync(tag, mode, plan.Winners.Select(w => w.Row).ToList(), context, ct).ConfigureAwait(false)
|
||||
: new Dictionary<long, UpsertOutcome>();
|
||||
planned = HistoryUpdatePlanner.Complete(plan, mode, outcome);
|
||||
}
|
||||
@@ -109,9 +118,11 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct)
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
if (!HistorianTagNames.IsValid(tag)) return HistorianStatusCodes.BadInvalidArgument;
|
||||
if (!await _tags.ExistsAsync(tag, ct).ConfigureAwait(false))
|
||||
return HistoryUpdatePlanner.DeleteRawResult(tagKnown: false, 0);
|
||||
@@ -123,17 +134,27 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
while (cursor < end)
|
||||
{
|
||||
var page = new List<DateTime>(BatchSize);
|
||||
var priors = new List<LiveRow>(BatchSize);
|
||||
await using (var conn = await _conn.OpenAsync(ct).ConfigureAwait(false))
|
||||
await using (var cmd = _conn.CreateCommand(conn,
|
||||
$"SELECT ts FROM {_conn.SampleTable} WHERE tag = $1 AND ts >= $2 AND ts < $3 AND NOT deleted ORDER BY ts LIMIT {BatchSize}"))
|
||||
$"SELECT {LiveColumns} FROM {_conn.SampleTable} WHERE tag = $1 AND ts >= $2 AND ts < $3 AND NOT deleted ORDER BY ts LIMIT {BatchSize}"))
|
||||
{
|
||||
QuestDbConnection.BindText(cmd, tag);
|
||||
QuestDbConnection.BindTimestamp(cmd, cursor);
|
||||
QuestDbConnection.BindTimestamp(cmd, end);
|
||||
await using var rd = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
while (await rd.ReadAsync(ct).ConfigureAwait(false)) page.Add(QuestDbConnection.ReadUtc(rd, 0));
|
||||
while (await rd.ReadAsync(ct).ConfigureAwait(false))
|
||||
{
|
||||
var row = ReadLiveRow(rd);
|
||||
page.Add(row.TimeUtc);
|
||||
priors.Add(row);
|
||||
}
|
||||
}
|
||||
if (page.Count == 0) break;
|
||||
// Trail BEFORE the change (no multi-table transaction): a crash here leaves an orphan trail row, never a
|
||||
// tombstone without its trail row.
|
||||
await WriteTrailAsync(tag, HistoryModificationType.Delete,
|
||||
priors.Select(r => (r.TimeUtc, (LiveRow?)r)).ToList(), context, ct).ConfigureAwait(false);
|
||||
await WriteTombstonesAsync(tag, page, ct).ConfigureAwait(false);
|
||||
// The wait between pages is required: without it the next page could re-read rows just tombstoned.
|
||||
await _wal.WaitAsync(_conn.SampleTable, ct).ConfigureAwait(false);
|
||||
@@ -145,10 +166,12 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct)
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc,
|
||||
HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(timesUtc);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
if (!HistorianTagNames.IsValid(tag))
|
||||
{
|
||||
var refused = new uint[timesUtc.Count];
|
||||
@@ -168,9 +191,14 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
|
||||
using var _ = await LockAsync(tag, ct).ConfigureAwait(false);
|
||||
var live = await ReadLiveAsync(tag, candidates.Select(t => new DateTime(t, DateTimeKind.Utc)).ToList(), ct).ConfigureAwait(false);
|
||||
var toDelete = candidates.Where(live.Contains).OrderBy(x => x).ToList();
|
||||
var toDelete = candidates.Where(live.ContainsKey).OrderBy(x => x).ToList();
|
||||
foreach (var chunk in toDelete.Chunk(BatchSize))
|
||||
{
|
||||
// Trail BEFORE the change (see the class remarks).
|
||||
await WriteTrailAsync(tag, HistoryModificationType.Delete,
|
||||
chunk.Select(t => (live[t].TimeUtc, (LiveRow?)live[t])).ToList(), context, ct).ConfigureAwait(false);
|
||||
await WriteTombstonesAsync(tag, chunk.Select(t => new DateTime(t, DateTimeKind.Utc)).ToList(), ct).ConfigureAwait(false);
|
||||
}
|
||||
if (toDelete.Count > 0) await _wal.WaitAsync(_conn.SampleTable, ct).ConfigureAwait(false);
|
||||
deleted.UnionWith(toDelete);
|
||||
return HistoryUpdatePlanner.DeleteAtTimeResults(rounded, deleted, tagKnown: true);
|
||||
@@ -183,48 +211,162 @@ internal sealed class QuestDbHistorianDataUpdater : IHistorianDataUpdater
|
||||
}
|
||||
|
||||
private async Task<Dictionary<long, UpsertOutcome>> ApplyAsync(string tag, HistoryUpdateMode mode,
|
||||
List<HistorianSampleRow> rows, CancellationToken ct)
|
||||
List<HistorianSampleRow> rows, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
// Key = the planner's tick (what Complete looks up); the stored time is Round(of the same row).
|
||||
var live = await ReadLiveAsync(tag, rows.Select(r => QuestDbConnection.Round(r.TimeUtc)).ToList(), ct).ConfigureAwait(false);
|
||||
var outcome = new Dictionary<long, UpsertOutcome>();
|
||||
var toWrite = new List<HistorianSampleRow>();
|
||||
var trail = new List<(DateTime, LiveRow?)>();
|
||||
foreach (var r in rows)
|
||||
{
|
||||
var present = live.Contains(QuestDbConnection.Round(r.TimeUtc).Ticks);
|
||||
var stored = QuestDbConnection.Round(r.TimeUtc);
|
||||
var present = live.TryGetValue(stored.Ticks, out var prior);
|
||||
if (mode == HistoryUpdateMode.Insert && present) continue;
|
||||
if (mode == HistoryUpdateMode.Replace && !present) continue;
|
||||
outcome[r.TimeUtc.Ticks] = present ? UpsertOutcome.Replaced : UpsertOutcome.Inserted;
|
||||
toWrite.Add(r);
|
||||
trail.Add((stored, present ? prior : null));
|
||||
}
|
||||
// Trail BEFORE the change (see the class remarks).
|
||||
await WriteTrailAsync(tag, ToModificationType(mode), trail, context, ct).ConfigureAwait(false);
|
||||
foreach (var chunk in toWrite.Chunk(BatchSize))
|
||||
await WriteRowsAsync(tag, chunk, ct).ConfigureAwait(false);
|
||||
if (toWrite.Count > 0) await _wal.WaitAsync(_conn.SampleTable, ct).ConfigureAwait(false);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
/// <summary>Ticks of the live (present, not deleted) rows near the given times: one range query per cluster (I-2).</summary>
|
||||
private async Task<HashSet<long>> ReadLiveAsync(string tag, IReadOnlyList<DateTime> times, CancellationToken ct)
|
||||
/// <summary>The live (present, not deleted) rows near the given times, keyed by stored ticks: one range query per cluster (I-2).</summary>
|
||||
private async Task<Dictionary<long, LiveRow>> ReadLiveAsync(string tag, IReadOnlyList<DateTime> times, CancellationToken ct)
|
||||
{
|
||||
var live = new HashSet<long>();
|
||||
var live = new Dictionary<long, LiveRow>();
|
||||
await using var conn = await _conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
foreach (var (lo, hi) in QuestDbTimeClusters.Cluster(times, QuestDbTimeClusters.ExistenceGap))
|
||||
{
|
||||
await using var cmd = _conn.CreateCommand(conn,
|
||||
$"SELECT ts FROM {_conn.SampleTable} WHERE tag = $1 AND ts >= $2 AND ts <= $3 AND NOT deleted");
|
||||
$"SELECT {LiveColumns} FROM {_conn.SampleTable} WHERE tag = $1 AND ts >= $2 AND ts <= $3 AND NOT deleted");
|
||||
QuestDbConnection.BindText(cmd, tag);
|
||||
QuestDbConnection.BindTimestamp(cmd, lo);
|
||||
QuestDbConnection.BindTimestamp(cmd, hi);
|
||||
await using var rd = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
while (await rd.ReadAsync(ct).ConfigureAwait(false))
|
||||
{
|
||||
live.Add(QuestDbConnection.ReadUtc(rd, 0).Ticks);
|
||||
var row = ReadLiveRow(rd);
|
||||
live[row.TimeUtc.Ticks] = row;
|
||||
Interlocked.Increment(ref _liveRowsRead);
|
||||
}
|
||||
}
|
||||
return live;
|
||||
}
|
||||
|
||||
/// <summary>The columns of a live sample the trail needs (prior value + status + server time).</summary>
|
||||
private const string LiveColumns = "ts, kind, num_value, int_value, str_value, status, server_ts";
|
||||
|
||||
/// <summary>A live sample as it was before a modification.</summary>
|
||||
private readonly record struct LiveRow(DateTime TimeUtc, HistorianValue Value, uint Status, DateTime? ServerTimeUtc);
|
||||
|
||||
private static LiveRow ReadLiveRow(NpgsqlDataReader rd)
|
||||
{
|
||||
var value = QuestDbHistorianDataSource.ToValue(
|
||||
QuestDbConnection.ReadKind(rd, 1),
|
||||
rd.IsDBNull(2) ? null : rd.GetDouble(2),
|
||||
rd.IsDBNull(3) ? null : rd.GetInt64(3),
|
||||
rd.IsDBNull(4) ? null : rd.GetString(4));
|
||||
var status = rd.IsDBNull(5) ? HistorianStatusCodes.BadNoData : SampleColumnMapper.FromLongStatus(rd.GetInt64(5));
|
||||
return new LiveRow(QuestDbConnection.ReadUtc(rd, 0), value, status,
|
||||
rd.IsDBNull(6) ? null : QuestDbConnection.ReadUtc(rd, 6));
|
||||
}
|
||||
|
||||
/// <summary>The Part 11 modification type recorded for an update mode.</summary>
|
||||
private static HistoryModificationType ToModificationType(HistoryUpdateMode mode) => mode switch
|
||||
{
|
||||
HistoryUpdateMode.Insert => HistoryModificationType.Insert,
|
||||
HistoryUpdateMode.Replace => HistoryModificationType.Replace,
|
||||
_ => HistoryModificationType.Update,
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// A process-wide, strictly increasing modification id (≈ UTC ticks, bumped past the last issued value). It orders a
|
||||
/// tag's trail within one source time; ids from two nodes interleave by wall clock, which is the modification order.
|
||||
/// </summary>
|
||||
internal static long NextModifiedId()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var last = Interlocked.Read(ref _lastModifiedId);
|
||||
var next = Math.Max(DateTime.UtcNow.Ticks, last + 1);
|
||||
if (Interlocked.CompareExchange(ref _lastModifiedId, next, last) == last) return next;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Appends one modification-trail row per entry (<c>prior = null</c> ⇒ an insert: kind Empty, NULL value and
|
||||
/// status), then waits for the trail's WAL visibility so a modified read right after the update sees it.
|
||||
/// </summary>
|
||||
private async Task WriteTrailAsync(string tag, HistoryModificationType type, IReadOnlyList<(DateTime TimeUtc, LiveRow? Prior)> entries,
|
||||
HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
if (entries.Count == 0) return;
|
||||
var now = DateTime.UtcNow;
|
||||
foreach (var chunk in entries.Chunk(BatchSize))
|
||||
{
|
||||
await using var conn = await _conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
await using var cmd = _conn.CreateCommand(conn, "");
|
||||
var sb = new System.Text.StringBuilder(
|
||||
$"INSERT INTO {_conn.SampleModifiedTable} (ts, tag, modified_id, modification_ts, update_type, principal, " +
|
||||
"api_key_id, kind, num_value, int_value, str_value, status, server_ts) VALUES ");
|
||||
for (var i = 0; i < chunk.Length; i++)
|
||||
{
|
||||
var (time, prior) = chunk[i];
|
||||
QuestDbConnection.BindTimestamp(cmd, time, clampToEpoch: true);
|
||||
QuestDbConnection.BindText(cmd, tag);
|
||||
QuestDbConnection.BindLong(cmd, NextModifiedId());
|
||||
QuestDbConnection.BindTimestamp(cmd, now);
|
||||
QuestDbConnection.BindByte(cmd, (byte)type);
|
||||
QuestDbConnection.BindText(cmd, context.StoredPrincipal);
|
||||
QuestDbConnection.BindText(cmd, context.StoredApiKeyId);
|
||||
if (prior is { } p)
|
||||
{
|
||||
var c = SampleColumnMapper.ToColumns(p.Value);
|
||||
QuestDbConnection.BindByte(cmd, c.ValueKind);
|
||||
Typed(cmd, NpgsqlDbType.Double, c.NumValue);
|
||||
Typed(cmd, NpgsqlDbType.Bigint, c.IntValue);
|
||||
Typed(cmd, NpgsqlDbType.Varchar, c.StrValue);
|
||||
QuestDbConnection.BindLong(cmd, SampleColumnMapper.ToLongStatus(p.Status));
|
||||
Typed(cmd, NpgsqlDbType.TimestampTz, p.ServerTimeUtc);
|
||||
}
|
||||
else
|
||||
{
|
||||
// No prior value: QuestDB BYTE has no NULL, so kind is Empty (0) and the NULL status marks "none".
|
||||
QuestDbConnection.BindByte(cmd, (byte)HistorianValueKind.Empty);
|
||||
Typed(cmd, NpgsqlDbType.Double, null);
|
||||
Typed(cmd, NpgsqlDbType.Bigint, null);
|
||||
Typed(cmd, NpgsqlDbType.Varchar, null);
|
||||
Typed(cmd, NpgsqlDbType.Bigint, null);
|
||||
Typed(cmd, NpgsqlDbType.TimestampTz, null);
|
||||
}
|
||||
|
||||
var b = i * 13;
|
||||
if (i > 0) sb.Append(',');
|
||||
sb.Append('(');
|
||||
for (var q = 1; q <= 13; q++) sb.Append(q > 1 ? "," : "").Append('$').Append(b + q);
|
||||
sb.Append(')');
|
||||
}
|
||||
cmd.CommandText = sb.ToString();
|
||||
try
|
||||
{
|
||||
await cmd.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
_log.LogWarning(ex, "QuestDB modification-trail INSERT of {Count} rows into {Table} failed; the update was not applied.",
|
||||
chunk.Length, _conn.SampleModifiedTable);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
await _wal.WaitAsync(_conn.SampleModifiedTable, ct).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
private Task WriteRowsAsync(string tag, IReadOnlyList<HistorianSampleRow> rows, CancellationToken ct) =>
|
||||
InsertAsync(rows.Count, ct, (cmd, i) =>
|
||||
{
|
||||
|
||||
@@ -6,7 +6,7 @@ using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb;
|
||||
|
||||
/// <summary>
|
||||
/// Lazily creates the three tables once per instance (<c>CREATE TABLE IF NOT EXISTS</c>), then reconciles the TTL
|
||||
/// Lazily creates the four tables once per instance (<c>CREATE TABLE IF NOT EXISTS</c>), then reconciles the TTL
|
||||
/// of the sample and event tables with <see cref="QuestDbHistorianOptions.RetentionDays"/>.
|
||||
/// <list type="bullet">
|
||||
/// <item><b>Concurrency:</b> two nodes starting together are safe. If a CREATE still fails with a server error
|
||||
|
||||
@@ -15,6 +15,8 @@ internal static class QuestDbTables
|
||||
public const string EventBase = "event";
|
||||
/// <summary>Base name of the tag catalog table.</summary>
|
||||
public const string TagBase = "tag";
|
||||
/// <summary>Base name of the Part 11 modification-trail table (#668).</summary>
|
||||
public const string SampleModifiedBase = "sample_modified";
|
||||
|
||||
/// <summary>The fixed designated timestamp of every <c>tag</c> row, which makes the dedup key an upsert by name.</summary>
|
||||
public static readonly DateTime TagEpoch = DateTime.UnixEpoch;
|
||||
@@ -25,12 +27,17 @@ internal static class QuestDbTables
|
||||
public static string Event(string prefix) => Name(prefix, EventBase);
|
||||
/// <summary>The prefixed tag table name.</summary>
|
||||
public static string Tag(string prefix) => Name(prefix, TagBase);
|
||||
/// <summary>The prefixed modification-trail table name.</summary>
|
||||
public static string SampleModified(string prefix) => Name(prefix, SampleModifiedBase);
|
||||
|
||||
/// <summary>All three table names in creation order.</summary>
|
||||
public static string[] All(string prefix) => [Sample(prefix), Event(prefix), Tag(prefix)];
|
||||
/// <summary>All four table names in creation order.</summary>
|
||||
public static string[] All(string prefix) => [Sample(prefix), Event(prefix), Tag(prefix), SampleModified(prefix)];
|
||||
|
||||
/// <summary>The tables that carry a TTL (sample and event; the tag catalog never expires).</summary>
|
||||
public static string[] Expiring(string prefix) => [Sample(prefix), Event(prefix)];
|
||||
/// <summary>
|
||||
/// The tables that carry a TTL (sample, event and the modification trail, which ages out with the samples it
|
||||
/// describes; the tag catalog never expires).
|
||||
/// </summary>
|
||||
public static string[] Expiring(string prefix) => [Sample(prefix), Event(prefix), SampleModified(prefix)];
|
||||
|
||||
/// <summary><c>CREATE TABLE IF NOT EXISTS</c> for the sample table.</summary>
|
||||
public static string CreateSample(string prefix, int? retentionDays) =>
|
||||
@@ -50,9 +57,22 @@ internal static class QuestDbTables
|
||||
$"CREATE TABLE IF NOT EXISTS {Tag(prefix)} (ts TIMESTAMP, name SYMBOL, data_type BYTE, eng_unit VARCHAR, " +
|
||||
"description VARCHAR, updated_utc TIMESTAMP) TIMESTAMP(ts) PARTITION BY YEAR WAL DEDUP UPSERT KEYS(ts, name)";
|
||||
|
||||
/// <summary>The three CREATE statements in creation order.</summary>
|
||||
/// <summary>
|
||||
/// <c>CREATE TABLE IF NOT EXISTS</c> for the Part 11 modification trail (#668): one row per sample a HistoryUpdate
|
||||
/// inserted, replaced or deleted, carrying the PRIOR value (NULL kind/value/status for an insert). <c>ts</c> is the
|
||||
/// sample's source time (so the TTL ages it out with the samples); <c>modified_id</c> orders the trail within one
|
||||
/// source time. Append-only: no DEDUP, never updated or deleted.
|
||||
/// </summary>
|
||||
public static string CreateSampleModified(string prefix, int? retentionDays) =>
|
||||
$"CREATE TABLE IF NOT EXISTS {SampleModified(prefix)} (ts TIMESTAMP, tag SYMBOL, modified_id LONG, " +
|
||||
"modification_ts TIMESTAMP, update_type BYTE, principal VARCHAR, api_key_id VARCHAR, kind BYTE, num_value DOUBLE, " +
|
||||
"int_value LONG, str_value VARCHAR, status LONG, server_ts TIMESTAMP) " +
|
||||
$"TIMESTAMP(ts) PARTITION BY DAY{Ttl(retentionDays)} WAL";
|
||||
|
||||
/// <summary>The four CREATE statements in creation order (parallel to <see cref="All"/>).</summary>
|
||||
public static string[] CreateAll(string prefix, int? retentionDays) =>
|
||||
[CreateSample(prefix, retentionDays), CreateEvent(prefix, retentionDays), CreateTag(prefix)];
|
||||
[CreateSample(prefix, retentionDays), CreateEvent(prefix, retentionDays), CreateTag(prefix),
|
||||
CreateSampleModified(prefix, retentionDays)];
|
||||
|
||||
/// <summary><c>ALTER TABLE t SET TTL n DAYS</c>; <c>null</c> removes the TTL (<c>SET TTL 0 DAYS</c>).</summary>
|
||||
public static string AlterTtl(string table, int? retentionDays) =>
|
||||
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
-- OtOpcUa SQL Server historian backend - schema version 2: OPC UA Part 11 modified-value history (Gitea #668).
|
||||
-- $(Schema) is substituted by SqlServerSchemaBootstrapper (identifier-validated) BEFORE the script is split on GO lines.
|
||||
-- Every statement is guarded so a re-run is a no-op.
|
||||
--
|
||||
-- One row per sample a HistoryUpdate inserted, replaced or deleted, carrying the PRIOR value (all NULL for an
|
||||
-- insert). Rows are written by the SAME statement that changes hist.Sample (OUTPUT ... INTO), so a failed update
|
||||
-- leaves no audit row and a committed one always has its row.
|
||||
-- WARNING: OUTPUT ... INTO forbids, on its target table: enabled triggers, either side of a FOREIGN KEY, and CHECK
|
||||
-- constraints. Do not add any of those to SampleModified, or every HistoryUpdate fails.
|
||||
IF OBJECT_ID(N'[$(Schema)].[SampleModified]', N'U') IS NULL
|
||||
CREATE TABLE [$(Schema)].[SampleModified] (
|
||||
TagId int NOT NULL,
|
||||
SourceTimeUtc datetime2(7) NOT NULL,
|
||||
ModifiedId bigint IDENTITY(1,1) NOT NULL,
|
||||
ModificationTimeUtc datetime2(7) NOT NULL,
|
||||
UpdateType tinyint NOT NULL,
|
||||
Principal nvarchar(200) NOT NULL,
|
||||
ApiKeyId nvarchar(200) NULL,
|
||||
ValueKind tinyint NULL,
|
||||
NumValue float NULL,
|
||||
IntValue bigint NULL,
|
||||
StrValue nvarchar(max) NULL,
|
||||
StatusCode int NULL,
|
||||
ServerTimeUtc datetime2(7) NULL,
|
||||
CONSTRAINT PK_SampleModified PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc, ModifiedId))
|
||||
GO
|
||||
IF NOT EXISTS (SELECT 1 FROM [$(Schema)].[SchemaVersion] WHERE Version = 2)
|
||||
INSERT [$(Schema)].[SchemaVersion] (Version, AppliedUtc) VALUES (2, SYSUTCDATETIME())
|
||||
GO
|
||||
+91
@@ -132,6 +132,97 @@ public sealed class SqlServerHistorianDataSource : IHistorianDataSource
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------ Modified (Part 11, #668)
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <remarks>
|
||||
/// Reads <c>SampleModified</c> (schema v2), the trail <see cref="SqlServerHistorianDataUpdater"/> writes in the
|
||||
/// same statement as each change. Windows follow <see cref="ReadRawAsync"/>; rows are ordered by
|
||||
/// <c>(SourceTimeUtc, ModifiedId)</c> (the clustered key after <c>TagId</c>, so a seek). <c>maxValues + 1</c> rows
|
||||
/// are fetched to report <see cref="HistoryModifiedReadResult.HasMore"/> exactly. An insert's audit row carries no
|
||||
/// prior value and is served <c>null</c> / <c>BadNoData</c>; its server timestamp is the modification time. An
|
||||
/// unknown tag is an empty result.
|
||||
/// </remarks>
|
||||
public async Task<HistoryModifiedReadResult> ReadModifiedAsync(
|
||||
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValues, HistoryModifiedCursor? resumeAfter,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(fullReference);
|
||||
var bounded = maxValues > 0 && maxValues < int.MaxValue;
|
||||
try
|
||||
{
|
||||
var values = new List<HistoryModifiedValue>();
|
||||
var hasMore = false;
|
||||
var tagId = await ResolveAsync(fullReference, cancellationToken).ConfigureAwait(false);
|
||||
if (tagId is { } id)
|
||||
{
|
||||
var reverse = startUtc > endUtc;
|
||||
string where;
|
||||
if (startUtc == endUtc) where = "SourceTimeUtc = @lo";
|
||||
else if (!reverse) where = "SourceTimeUtc >= @lo AND SourceTimeUtc < @hi";
|
||||
else where = "SourceTimeUtc > @lo AND SourceTimeUtc <= @hi";
|
||||
if (resumeAfter is not null)
|
||||
where += reverse
|
||||
? " AND (SourceTimeUtc < @ct OR (SourceTimeUtc = @ct AND ModifiedId < @cid))"
|
||||
: " AND (SourceTimeUtc > @ct OR (SourceTimeUtc = @ct AND ModifiedId > @cid))";
|
||||
var order = reverse ? "DESC" : "ASC";
|
||||
var (lo, hi) = startUtc <= endUtc ? (startUtc, endUtc) : (endUtc, startUtc);
|
||||
|
||||
await using var conn = await _conn.OpenAsync(cancellationToken).ConfigureAwait(false);
|
||||
await using var cmd = _conn.CreateCommand(conn,
|
||||
$"SELECT {(bounded ? "TOP (@take) " : "")}SourceTimeUtc, ValueKind, NumValue, IntValue, StrValue, StatusCode, " +
|
||||
"ServerTimeUtc, ModifiedId, ModificationTimeUtc, UpdateType, Principal " +
|
||||
$"FROM {_conn.Q("SampleModified")} WHERE TagId = @id AND {where} " +
|
||||
$"ORDER BY SourceTimeUtc {order}, ModifiedId {order}");
|
||||
if (bounded) cmd.Parameters.Add(new SqlParameter("@take", SqlDbType.BigInt) { Value = (long)maxValues + 1 });
|
||||
cmd.Parameters.Add(new SqlParameter("@id", SqlDbType.Int) { Value = id });
|
||||
SqlHistorianConnection.AddDateTime2(cmd, "@lo", lo);
|
||||
SqlHistorianConnection.AddDateTime2(cmd, "@hi", hi);
|
||||
if (resumeAfter is not null)
|
||||
{
|
||||
SqlHistorianConnection.AddDateTime2(cmd, "@ct", resumeAfter.SourceTimeUtc);
|
||||
cmd.Parameters.Add(new SqlParameter("@cid", SqlDbType.BigInt) { Value = resumeAfter.ModifiedId });
|
||||
}
|
||||
|
||||
await using var reader = await cmd.ExecuteReaderAsync(cancellationToken).ConfigureAwait(false);
|
||||
while (await reader.ReadAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
if (bounded && values.Count == maxValues)
|
||||
{
|
||||
hasMore = true;
|
||||
break;
|
||||
}
|
||||
|
||||
var sourceTime = SqlHistorianConnection.GetUtc(reader, 0);
|
||||
var modifiedAt = SqlHistorianConnection.GetUtc(reader, 8);
|
||||
DataValueSnapshot prior;
|
||||
if (reader.IsDBNull(1))
|
||||
{
|
||||
// An insert: there was no prior value.
|
||||
prior = new DataValueSnapshot(null, HistorianStatusCodes.BadNoData, sourceTime, modifiedAt);
|
||||
}
|
||||
else
|
||||
{
|
||||
var s = ReadStored(reader, 0);
|
||||
prior = new DataValueSnapshot(s.Value.ToObject(), s.Status, s.TimeUtc,
|
||||
reader.IsDBNull(6) ? modifiedAt : SqlHistorianConnection.GetUtc(reader, 6));
|
||||
}
|
||||
|
||||
values.Add(new HistoryModifiedValue(
|
||||
prior, modifiedAt, (HistoryModificationType)reader.GetByte(9), reader.GetString(10), reader.GetInt64(7)));
|
||||
}
|
||||
}
|
||||
|
||||
RecordSuccess();
|
||||
return new HistoryModifiedReadResult(values, hasMore);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
RecordFailure(ex);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------------------------ AtTime
|
||||
|
||||
/// <inheritdoc />
|
||||
|
||||
+54
-8
@@ -22,6 +22,11 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
||||
/// is written.</para>
|
||||
/// <para><b>Unknown tag.</b> Insert/Update create it via <see cref="SqlServerTagCatalog"/>; Replace and the
|
||||
/// deletes report <c>BadNoEntryExists</c> (DeleteRaw: <c>BadNoData</c>) without creating anything.</para>
|
||||
/// <para><b>Modified-value history (Part 11, #668).</b> Every row the MERGE inserts or updates and every row a
|
||||
/// delete removes is copied into <c>SampleModified</c> by an <c>OUTPUT … INTO</c> on the SAME statement — the prior
|
||||
/// value from <c>deleted.*</c> (NULL for an insert), the caller's <see cref="HistoryUpdateContext"/>, the
|
||||
/// <see cref="HistoryModificationType"/> and <c>SYSUTCDATETIME()</c>. One statement means the sample change and its
|
||||
/// audit row commit or roll back together: a failed update leaves no audit row.</para>
|
||||
/// </summary>
|
||||
internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
{
|
||||
@@ -39,10 +44,11 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<IReadOnlyList<uint>> UpdateDataAsync(string tag, HistoryUpdateMode mode,
|
||||
IReadOnlyList<HistorizationValue> values, CancellationToken ct)
|
||||
IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(values);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
var plan = HistoryUpdatePlanner.Plan(tag, mode, values);
|
||||
if (plan.Winners.Count == 0) return HistoryUpdatePlanner.Complete(plan, mode, new Dictionary<long, UpsertOutcome>());
|
||||
|
||||
@@ -52,14 +58,16 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
|
||||
var outcome = tagId is null
|
||||
? new Dictionary<long, UpsertOutcome>()
|
||||
: await MergeAsync(tagId.Value, mode, plan.Winners.Select(w => w.Row).ToList(), ct).ConfigureAwait(false);
|
||||
: await MergeAsync(tagId.Value, mode, plan.Winners.Select(w => w.Row).ToList(), context, ct).ConfigureAwait(false);
|
||||
return HistoryUpdatePlanner.Complete(plan, mode, outcome);
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct)
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
if (!HistorianTagNames.IsValid(tag)) return HistorianStatusCodes.BadInvalidArgument;
|
||||
var tagId = await _tags.TryGetTagIdAsync(tag, ct).ConfigureAwait(false);
|
||||
if (tagId is null) return HistoryUpdatePlanner.DeleteRawResult(tagKnown: false, 0);
|
||||
@@ -67,9 +75,11 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
await using var tx = (SqlTransaction)await conn.BeginTransactionAsync(ct).ConfigureAwait(false);
|
||||
int rows;
|
||||
await using (var cmd = _conn.CreateCommand(conn,
|
||||
$"DELETE FROM {_conn.Q("Sample")} WHERE TagId = @tagId AND SourceTimeUtc >= @s AND SourceTimeUtc < @e", tx))
|
||||
$"DELETE FROM {_conn.Q("Sample")} {AuditInto(HistoryModificationType.Delete)} " +
|
||||
"WHERE TagId = @tagId AND SourceTimeUtc >= @s AND SourceTimeUtc < @e", tx))
|
||||
{
|
||||
cmd.Parameters.Add(new SqlParameter("@tagId", System.Data.SqlDbType.Int) { Value = tagId.Value });
|
||||
AddAuditParameters(cmd, context);
|
||||
SqlHistorianConnection.AddDateTime2(cmd, "@s", startUtc);
|
||||
SqlHistorianConnection.AddDateTime2(cmd, "@e", endUtc);
|
||||
rows = await cmd.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
|
||||
@@ -79,10 +89,12 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct)
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(string tag, IReadOnlyList<DateTime> timesUtc,
|
||||
HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(tag);
|
||||
ArgumentNullException.ThrowIfNull(timesUtc);
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
if (!HistorianTagNames.IsValid(tag))
|
||||
{
|
||||
var refused = new uint[timesUtc.Count];
|
||||
@@ -100,9 +112,11 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
await using var conn = await _conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
await using var tx = (SqlTransaction)await conn.BeginTransactionAsync(ct).ConfigureAwait(false);
|
||||
await using (var cmd = _conn.CreateCommand(conn,
|
||||
$"DELETE s OUTPUT deleted.SourceTimeUtc FROM {_conn.Q("Sample")} AS s JOIN @times AS t ON s.SourceTimeUtc = t.T WHERE s.TagId = @tagId", tx))
|
||||
$"DELETE s {AuditInto(HistoryModificationType.Delete)} OUTPUT deleted.SourceTimeUtc " +
|
||||
$"FROM {_conn.Q("Sample")} AS s JOIN @times AS t ON s.SourceTimeUtc = t.T WHERE s.TagId = @tagId", tx))
|
||||
{
|
||||
cmd.Parameters.Add(new SqlParameter("@tagId", System.Data.SqlDbType.Int) { Value = tagId.Value });
|
||||
AddAuditParameters(cmd, context);
|
||||
_conn.AddStructured(cmd, "@times", "TimeRow",
|
||||
SampleRowMapper.ToTimeRecords(distinct.Select(t => new DateTime(t, DateTimeKind.Utc))));
|
||||
await using var rd = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
@@ -114,7 +128,7 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
}
|
||||
|
||||
private async Task<Dictionary<long, UpsertOutcome>> MergeAsync(int tagId, HistoryUpdateMode mode,
|
||||
List<HistorianSampleRow> rows, CancellationToken ct)
|
||||
List<HistorianSampleRow> rows, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
var sql = $"MERGE {_conn.Q("Sample")} WITH (HOLDLOCK) AS t USING @rows AS r ON t.TagId = @tagId AND t.SourceTimeUtc = r.SourceTimeUtc ";
|
||||
if (mode != HistoryUpdateMode.Insert)
|
||||
@@ -123,7 +137,9 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
if (mode != HistoryUpdateMode.Replace)
|
||||
sql += "WHEN NOT MATCHED THEN INSERT (TagId, SourceTimeUtc, ValueKind, NumValue, IntValue, StrValue, StatusCode, ServerTimeUtc) " +
|
||||
"VALUES (@tagId, r.SourceTimeUtc, r.ValueKind, r.NumValue, r.IntValue, r.StrValue, r.StatusCode, SYSUTCDATETIME()) ";
|
||||
sql += "OUTPUT $action, r.SourceTimeUtc;";
|
||||
// The MERGE's deleted.* is NULL for an inserted row, so one OUTPUT INTO records both an insert (no prior value)
|
||||
// and a replace (the prior value). TagId comes from inserted.* — present on both branches.
|
||||
sql += $"{AuditInto(ToModificationType(mode), "inserted")} OUTPUT $action, r.SourceTimeUtc;";
|
||||
|
||||
var outcome = new Dictionary<long, UpsertOutcome>();
|
||||
await using var conn = await _conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
@@ -131,6 +147,7 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
await using (var cmd = _conn.CreateCommand(conn, sql, tx))
|
||||
{
|
||||
cmd.Parameters.Add(new SqlParameter("@tagId", System.Data.SqlDbType.Int) { Value = tagId });
|
||||
AddAuditParameters(cmd, context);
|
||||
_conn.AddStructured(cmd, "@rows", "SampleRow", SampleRowMapper.ToSampleRecords(rows));
|
||||
await using var rd = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
while (await rd.ReadAsync(ct).ConfigureAwait(false))
|
||||
@@ -141,4 +158,33 @@ internal sealed class SqlServerHistorianDataUpdater : IHistorianDataUpdater
|
||||
await tx.CommitAsync(ct).ConfigureAwait(false);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
/// <summary>The Part 11 modification type recorded for an update mode.</summary>
|
||||
/// <param name="mode">The update mode.</param>
|
||||
/// <returns>Insert ⇒ Insert, Replace ⇒ Replace, Update ⇒ Update.</returns>
|
||||
internal static HistoryModificationType ToModificationType(HistoryUpdateMode mode) => mode switch
|
||||
{
|
||||
HistoryUpdateMode.Insert => HistoryModificationType.Insert,
|
||||
HistoryUpdateMode.Replace => HistoryModificationType.Replace,
|
||||
_ => HistoryModificationType.Update,
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// The <c>OUTPUT … INTO SampleModified</c> clause copying each affected row's prior value (<c>deleted.*</c>) into
|
||||
/// the modification trail. <paramref name="keySource"/> names the pseudo-table that carries <c>TagId</c> /
|
||||
/// <c>SourceTimeUtc</c> for every affected row (<c>deleted</c> for a DELETE, <c>inserted</c> for the MERGE).
|
||||
/// </summary>
|
||||
private string AuditInto(HistoryModificationType type, string keySource = "deleted") =>
|
||||
$"OUTPUT {keySource}.TagId, {keySource}.SourceTimeUtc, SYSUTCDATETIME(), {(byte)type}, @principal, @apiKeyId, " +
|
||||
"deleted.ValueKind, deleted.NumValue, deleted.IntValue, deleted.StrValue, deleted.StatusCode, deleted.ServerTimeUtc " +
|
||||
$"INTO {_conn.Q("SampleModified")} (TagId, SourceTimeUtc, ModificationTimeUtc, UpdateType, Principal, ApiKeyId, " +
|
||||
"ValueKind, NumValue, IntValue, StrValue, StatusCode, ServerTimeUtc)";
|
||||
|
||||
private static void AddAuditParameters(SqlCommand cmd, HistoryUpdateContext context)
|
||||
{
|
||||
cmd.Parameters.Add(new SqlParameter("@principal", System.Data.SqlDbType.NVarChar, HistoryUpdateContext.MaxIdentityLength)
|
||||
{ Value = context.StoredPrincipal });
|
||||
cmd.Parameters.Add(new SqlParameter("@apiKeyId", System.Data.SqlDbType.NVarChar, HistoryUpdateContext.MaxIdentityLength)
|
||||
{ Value = (object?)context.StoredApiKeyId ?? DBNull.Value });
|
||||
}
|
||||
}
|
||||
|
||||
+2
-1
@@ -18,7 +18,8 @@ public sealed partial class SqlServerHistorianOptions
|
||||
|
||||
/// <summary>
|
||||
/// <c>true</c> creates the schema and applies missing scripts on first use; <c>false</c> only verifies
|
||||
/// that schema version 1 exists and fails every historian call with a clear error otherwise.
|
||||
/// that the required schema version (<c>SqlServerSchemaBootstrapper.RequiredVersion</c>, currently 2) exists and
|
||||
/// fails every historian call with a clear error otherwise.
|
||||
/// </summary>
|
||||
public bool AutoCreateSchema { get; set; } = true;
|
||||
|
||||
|
||||
+13
-3
@@ -7,7 +7,7 @@ using ZB.MOM.WW.OtOpcUa.Runtime.Historian;
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
||||
|
||||
/// <summary>
|
||||
/// Deletes <c>hist.Sample</c> / <c>hist.Event</c> rows older than the retention window, in small
|
||||
/// Deletes <c>hist.Sample</c> / <c>hist.SampleModified</c> / <c>hist.Event</c> rows older than the retention window, in small
|
||||
/// <c>DELETE TOP(n)</c> batches (under the lock-escalation threshold). Samples are purged per tag so each
|
||||
/// delete is a seek on the clustered key <c>(TagId, SourceTimeUtc)</c>; events seek on <c>CX_Event_Time</c>.
|
||||
/// A session <c>sp_getapplock</c> (timeout 0) serialises purgers that share one database; a loser skips.
|
||||
@@ -53,7 +53,7 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
||||
return [];
|
||||
}
|
||||
|
||||
long samples = 0;
|
||||
long samples = 0, modified = 0;
|
||||
var tagIds = new List<int>();
|
||||
await using (var cmd = _conn.CreateCommand(conn, $"SELECT TagId FROM {_conn.Q("Tag")}"))
|
||||
await using (var r = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false))
|
||||
@@ -64,13 +64,23 @@ internal sealed class SqlServerHistorianRetention : IHistorianRetention
|
||||
samples += await DeleteLoopAsync(conn,
|
||||
$"DELETE TOP (@n) FROM {_conn.Q("Sample")} WHERE TagId = @id AND SourceTimeUtc < @cutoff",
|
||||
cutoff, id, ct).ConfigureAwait(false);
|
||||
// The Part 11 modification trail (#668) ages out on the SAME source-time cutoff as the samples it
|
||||
// describes; a purge never writes to it (no trigger, no OUTPUT INTO).
|
||||
modified += await DeleteLoopAsync(conn,
|
||||
$"DELETE TOP (@n) FROM {_conn.Q("SampleModified")} WHERE TagId = @id AND SourceTimeUtc < @cutoff",
|
||||
cutoff, id, ct).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
var events = await DeleteLoopAsync(conn,
|
||||
$"DELETE TOP (@n) FROM {_conn.Q("Event")} WHERE EventTimeUtc < @cutoff",
|
||||
cutoff, null, ct).ConfigureAwait(false);
|
||||
|
||||
return [new RetentionPurgeResult("Sample", samples), new RetentionPurgeResult("Event", events)];
|
||||
return
|
||||
[
|
||||
new RetentionPurgeResult("Sample", samples),
|
||||
new RetentionPurgeResult("SampleModified", modified),
|
||||
new RetentionPurgeResult("Event", events),
|
||||
];
|
||||
}
|
||||
finally
|
||||
{
|
||||
|
||||
+9
-5
@@ -14,12 +14,16 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer;
|
||||
/// <see cref="BootstrapFailureCache.DefaultDuration"/> retries (<see cref="BootstrapFailureCache"/>). Within that window the last failure is re-thrown without touching
|
||||
/// SQL, so a missing schema (or a SQL outage) does not serialise every write, read and alarm event behind one
|
||||
/// semaphore and one round-trip each.
|
||||
/// With <c>AutoCreateSchema=false</c> it only verifies <c>MAX(Version) >= 1</c>.
|
||||
/// With <c>AutoCreateSchema=false</c> it only verifies <c>MAX(Version) >= </c><see cref="RequiredVersion"/>.
|
||||
/// </summary>
|
||||
internal sealed partial class SqlServerSchemaBootstrapper
|
||||
{
|
||||
/// <summary>The schema version this build requires.</summary>
|
||||
public const int RequiredVersion = 1;
|
||||
/// <summary>
|
||||
/// The schema version this build requires: 2 = <c>002_sample_modified.sql</c> (Part 11 modified-value
|
||||
/// history, Gitea #668). An existing v1 schema is upgraded in place on the next bootstrap when
|
||||
/// <c>AutoCreateSchema</c> is on; otherwise the script must be applied by hand.
|
||||
/// </summary>
|
||||
public const int RequiredVersion = 2;
|
||||
|
||||
private const string LockResource = "OtOpcUaHistorianSchema";
|
||||
private const int LockTimeoutMs = 30_000;
|
||||
@@ -48,7 +52,7 @@ internal sealed partial class SqlServerSchemaBootstrapper
|
||||
/// <summary>Ensures the schema is ready; no-op after the first success.</summary>
|
||||
/// <param name="ct">Cancellation.</param>
|
||||
/// <returns>A task completing when the schema is usable.</returns>
|
||||
/// <exception cref="HistorianSchemaMissingException">AutoCreateSchema is false and version 1 is absent.</exception>
|
||||
/// <exception cref="HistorianSchemaMissingException">AutoCreateSchema is false and <see cref="RequiredVersion"/> is absent.</exception>
|
||||
public async Task EnsureAsync(CancellationToken ct)
|
||||
{
|
||||
if (_ready) return;
|
||||
@@ -94,7 +98,7 @@ internal sealed partial class SqlServerSchemaBootstrapper
|
||||
if (v < RequiredVersion)
|
||||
throw new HistorianSchemaMissingException(
|
||||
$"Historian schema [{_conn.Schema}] is missing version {RequiredVersion} (found {v}) and AutoCreateSchema is false. " +
|
||||
"Apply docs/historian/sqlserver/001_initial.sql or enable AutoCreateSchema.");
|
||||
"Apply the docs/historian/sqlserver/NNN_*.sql scripts above version " + v + " in order, or enable AutoCreateSchema.");
|
||||
}
|
||||
|
||||
private async Task ApplyAsync(SqlConnection conn, CancellationToken ct)
|
||||
|
||||
@@ -31,11 +31,20 @@ namespace ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
/// prior pages and must be dropped from the head of the next page (tie de-dup).
|
||||
/// </param>
|
||||
/// <param name="NumValuesPerNode">The client's per-page cap; re-applied to every resumed page.</param>
|
||||
/// <param name="IsEvents">
|
||||
/// <c>true</c> for an EVENT HistoryRead cursor (Gitea #669): <paramref name="Tagname"/> then carries the
|
||||
/// registered historian event-source name and the boundary fields are over <c>EventTimeUtc</c>. A cursor
|
||||
/// is only ever resumed by the read kind that issued it, so a raw point presented to an events read
|
||||
/// (or vice versa) is a <c>BadContinuationPointInvalid</c> miss.
|
||||
/// <param name="Kind">
|
||||
/// Which read issued the cursor (Gitea #669 / #668). A cursor is only ever resumed by the read kind that issued it, so a
|
||||
/// raw point presented to an events or modified read (or any other mismatch) is a <c>BadContinuationPointInvalid</c>
|
||||
/// miss. <see cref="HistoryCursorKind.Events"/>: <paramref name="Tagname"/> carries the registered historian
|
||||
/// event-source name and the boundary fields are over <c>EventTimeUtc</c>.
|
||||
/// </param>
|
||||
/// <param name="LastModifiedId">
|
||||
/// <see cref="HistoryCursorKind.Modified"/> only: the <c>ModifiedId</c> of the last returned modification. With
|
||||
/// <paramref name="NextStartUtc"/> (its source time) it is the exclusive resume key; <paramref name="BoundarySkipCount"/>
|
||||
/// is unused because the key is unique.
|
||||
/// </param>
|
||||
/// <param name="WindowStartUtc">
|
||||
/// <see cref="HistoryCursorKind.Modified"/> only: the normalised window start, so a resumed page keeps the original
|
||||
/// window and direction (the modified read resumes by key, not by moving the window start).
|
||||
/// </param>
|
||||
internal sealed record HistoryContinuationState(
|
||||
string Tagname,
|
||||
@@ -43,7 +52,22 @@ internal sealed record HistoryContinuationState(
|
||||
DateTime EndUtc,
|
||||
int BoundarySkipCount,
|
||||
uint NumValuesPerNode,
|
||||
bool IsEvents = false);
|
||||
HistoryCursorKind Kind = HistoryCursorKind.Raw,
|
||||
long LastModifiedId = 0,
|
||||
DateTime WindowStartUtc = default);
|
||||
|
||||
/// <summary>The HistoryRead kind that issued a <see cref="HistoryContinuationState"/>.</summary>
|
||||
internal enum HistoryCursorKind
|
||||
{
|
||||
/// <summary>HistoryReadRaw (values).</summary>
|
||||
Raw,
|
||||
|
||||
/// <summary>HistoryReadEvents (#669).</summary>
|
||||
Events,
|
||||
|
||||
/// <summary>HistoryReadRaw with <c>IsReadModified = true</c> — the Part 11 modification trail (#668).</summary>
|
||||
Modified,
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Pure server-side continuation-point paging decisions for the count-capped variable-history arms
|
||||
|
||||
@@ -4977,8 +4977,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
{
|
||||
if (details.IsReadModified)
|
||||
{
|
||||
// We never serve modified-value history; mark this node unsupported and move on.
|
||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||
// Part 11 modified-value history (#668): the prior values every HistoryUpdate recorded. Gated by the
|
||||
// same HistoryRead authorization as a raw read (above); a backend with no trail ⇒ unsupported.
|
||||
work.Add(() => ServeModifiedPagedAsync(
|
||||
handle, session, nodesToRead, results, errors,
|
||||
details.StartTime, details.EndTime, details.NumValuesPerNode,
|
||||
ct));
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -5156,7 +5160,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// for THIS node only.
|
||||
var state = _historyContinuationStore.TryTake(session, inboundCp!);
|
||||
// A point is bound to the node that issued it: another node's source name is a miss too.
|
||||
if (state is null || !state.IsEvents || state.Tagname != (sourceName ?? string.Empty))
|
||||
if (state is null || state.Kind != HistoryCursorKind.Events || state.Tagname != (sourceName ?? string.Empty))
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadContinuationPointInvalid;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadContinuationPointInvalid };
|
||||
@@ -5237,7 +5241,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
if (nextStartUtc is { } resumeAt)
|
||||
{
|
||||
pendingCursor = new HistoryContinuationState(
|
||||
sourceName ?? string.Empty, resumeAt, endUtc, nextSkip, numValuesPerNode, IsEvents: true);
|
||||
sourceName ?? string.Empty, resumeAt, endUtc, nextSkip, numValuesPerNode, HistoryCursorKind.Events);
|
||||
}
|
||||
}
|
||||
else if (truncated && events.Count > 0)
|
||||
@@ -5247,7 +5251,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
var skip = 0;
|
||||
for (var i = events.Count - 1; i >= 0 && events[i].EventTimeUtc == boundary; i--) skip++;
|
||||
pendingCursor = new HistoryContinuationState(
|
||||
sourceName ?? string.Empty, boundary, endUtc, skip, numValuesPerNode, IsEvents: true);
|
||||
sourceName ?? string.Empty, boundary, endUtc, skip, numValuesPerNode, HistoryCursorKind.Events);
|
||||
}
|
||||
|
||||
var historyEvent = ProjectEvents(events, selectClauses);
|
||||
@@ -5374,7 +5378,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
}
|
||||
|
||||
var items = (IList<DataValue>?)details.UpdateValues ?? Array.Empty<DataValue>();
|
||||
return (updater, tagname, ct) => UpdateDataAsync(updater, tagname, mode, items, ct);
|
||||
return (updater, tagname, who, ct) => UpdateDataAsync(updater, tagname, mode, items, who, ct);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -5390,17 +5394,18 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
ServeHistoryUpdate(context, nodesToProcess, results, errors, handle =>
|
||||
{
|
||||
var details = nodesToUpdate[handle.Index];
|
||||
// We never keep modified-value history, so there is none to delete.
|
||||
// Deleting the modified-value trail itself (Part 11 IsDeleteModified) is not supported: the trail is the
|
||||
// audit record of every change and is aged out only by retention (#668).
|
||||
if (details.IsDeleteModified) return null;
|
||||
var start = details.StartTime;
|
||||
var end = details.EndTime;
|
||||
// Part 11 requires both bounds of a raw delete; DateTime.MinValue is the wire's "unspecified". Refuse
|
||||
// it here rather than let a backend read MinValue as "from the beginning of time".
|
||||
if (start == DateTime.MinValue || end == DateTime.MinValue)
|
||||
return (_, _, _) => Task.FromResult(new HistoryUpdateResult { StatusCode = StatusCodes.BadInvalidArgument });
|
||||
return async (updater, tagname, ct) =>
|
||||
return (_, _, _, _) => Task.FromResult(new HistoryUpdateResult { StatusCode = StatusCodes.BadInvalidArgument });
|
||||
return async (updater, tagname, who, ct) =>
|
||||
{
|
||||
var code = await updater.DeleteRawAsync(tagname, start, end, ct).ConfigureAwait(false);
|
||||
var code = await updater.DeleteRawAsync(tagname, start, end, who, ct).ConfigureAwait(false);
|
||||
return new HistoryUpdateResult { StatusCode = code };
|
||||
};
|
||||
});
|
||||
@@ -5418,11 +5423,11 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
ServeHistoryUpdate(context, nodesToProcess, results, errors, handle =>
|
||||
{
|
||||
var times = nodesToUpdate[handle.Index].ReqTimes?.ToList() ?? new List<DateTime>();
|
||||
return async (updater, tagname, ct) =>
|
||||
return async (updater, tagname, who, ct) =>
|
||||
{
|
||||
var codes = times.Count == 0
|
||||
? Array.Empty<uint>()
|
||||
: await updater.DeleteAtTimeAsync(tagname, times, ct).ConfigureAwait(false);
|
||||
: await updater.DeleteAtTimeAsync(tagname, times, who, ct).ConfigureAwait(false);
|
||||
if (codes.Count != times.Count)
|
||||
throw new InvalidOperationException(
|
||||
$"historian updater returned {codes.Count} result(s) for {times.Count} time(s)");
|
||||
@@ -5445,11 +5450,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
/// <param name="tagname">The node's historian tagname.</param>
|
||||
/// <param name="mode">The update mode.</param>
|
||||
/// <param name="items">The request's values, in order.</param>
|
||||
/// <param name="who">The caller, recorded in the modification trail.</param>
|
||||
/// <param name="ct">The request deadline.</param>
|
||||
/// <returns>The node's result, carrying one operation result per item.</returns>
|
||||
private static async Task<HistoryUpdateResult> UpdateDataAsync(
|
||||
IHistorianDataUpdater updater, string tagname, HistoryUpdateMode mode, IList<DataValue> items,
|
||||
CancellationToken ct)
|
||||
HistoryUpdateContext who, CancellationToken ct)
|
||||
{
|
||||
var codes = new uint[items.Count];
|
||||
var toSend = new List<HistorizationValue>(items.Count);
|
||||
@@ -5478,7 +5484,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
|
||||
if (toSend.Count > 0)
|
||||
{
|
||||
var backend = await updater.UpdateDataAsync(tagname, mode, toSend, ct).ConfigureAwait(false);
|
||||
var backend = await updater.UpdateDataAsync(tagname, mode, toSend, who, ct).ConfigureAwait(false);
|
||||
if (backend.Count != toSend.Count)
|
||||
throw new InvalidOperationException(
|
||||
$"historian updater returned {backend.Count} result(s) for {toSend.Count} value(s)");
|
||||
@@ -5510,11 +5516,14 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
List<NodeHandle> nodesToProcess,
|
||||
IList<HistoryUpdateResult> results,
|
||||
IList<ServiceResult> errors,
|
||||
Func<NodeHandle, Func<IHistorianDataUpdater, string, CancellationToken, Task<HistoryUpdateResult>>?> plan)
|
||||
Func<NodeHandle, Func<IHistorianDataUpdater, string, HistoryUpdateContext, CancellationToken, Task<HistoryUpdateResult>>?> plan)
|
||||
{
|
||||
nodesToProcess = AuthorizeHistoryUpdate(context, nodesToProcess, results, errors);
|
||||
if (nodesToProcess.Count == 0) return;
|
||||
|
||||
// Who is rewriting history — recorded on every modification-trail row (Part 11 ModificationInfo.UserName, #668).
|
||||
var who = HistoryUpdateCallerOf(context);
|
||||
|
||||
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||
{
|
||||
using var deadline = CreateDeadline();
|
||||
@@ -5533,13 +5542,30 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
|
||||
var h = handle;
|
||||
var tag = tagname;
|
||||
work.Add(() => ServeUpdateNodeAsync(h, results, errors, token => operation(updater!, tag, token), ct));
|
||||
work.Add(() => ServeUpdateNodeAsync(h, results, errors, token => operation(updater!, tag, who, token), ct));
|
||||
}
|
||||
|
||||
RunBounded(work, HistoryReadBatchParallelism);
|
||||
});
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The <see cref="HistoryUpdateContext"/> of the session behind <paramref name="context"/>: its API-key principal
|
||||
/// (or the reserved <c>anonymous</c> principal) and key id. Only an identity that already passed
|
||||
/// <see cref="AuthorizeHistoryUpdate"/> reaches a backend, so the anonymous fallback for a missing identity is
|
||||
/// defence in depth, not a path.
|
||||
/// </summary>
|
||||
/// <param name="context">The SDK context carrying the session identity.</param>
|
||||
/// <returns>The caller recorded in the modification trail.</returns>
|
||||
internal static HistoryUpdateContext HistoryUpdateCallerOf(ServerSystemContext context)
|
||||
{
|
||||
var identity = (context as ISessionOperationContext)?.UserIdentity as RoleCarryingUserIdentity
|
||||
?? context.OperationContext?.UserIdentity as RoleCarryingUserIdentity;
|
||||
return identity is null
|
||||
? new HistoryUpdateContext(OpcUaPrincipals.Anonymous, null)
|
||||
: new HistoryUpdateContext(identity.Principal, identity.ApiKeyId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Runs one node's HistoryUpdate operation and fills its slots. Never throws (the
|
||||
/// <see cref="RunBounded"/> contract): a deadline cancellation is <c>BadTimeout</c>, any backend failure
|
||||
@@ -5942,7 +5968,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
// Resume read: take the stored cursor. A miss (unknown / evicted / malformed point) ⇒
|
||||
// BadContinuationPointInvalid for THIS node.
|
||||
var state = _historyContinuationStore.TryTake(session, inboundCp);
|
||||
if (state is null || state.IsEvents
|
||||
if (state is null || state.Kind != HistoryCursorKind.Raw
|
||||
|| !TryGetHistorizedTagname(handle.NodeId, out var boundTag) || boundTag != state.Tagname)
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadContinuationPointInvalid;
|
||||
@@ -6108,6 +6134,138 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Serve one historized variable for a HistoryReadRaw with <c>IsReadModified = true</c> (Part 11 modified-value
|
||||
/// history, Gitea #668), with continuation-point paging. The backend orders the trail by
|
||||
/// <c>(SourceTimeUtc, ModifiedId)</c> and reports truncation itself (<see cref="HistoryModifiedReadResult.HasMore"/>),
|
||||
/// so the cursor is an exact, exclusive key — there are no ties to trim. Window normalisation matches
|
||||
/// <see cref="ServeRawPagedAsync"/>: with a cap, an unset EndTime pages forward to an open bound and an unset
|
||||
/// StartTime pages backward. A cursor is bound to the node's tagname and to the modified read kind; any other
|
||||
/// point is <c>BadContinuationPointInvalid</c>. A backend that keeps no trail (Gateway, Null) throws
|
||||
/// <see cref="NotSupportedException"/> ⇒ <c>BadHistoryOperationUnsupported</c> for this node.
|
||||
/// </summary>
|
||||
/// <param name="handle">The node handle; <c>handle.Index</c> indexes results/errors.</param>
|
||||
/// <param name="session">The session (null on the session-less in-process path ⇒ single-shot).</param>
|
||||
/// <param name="nodesToRead">The per-node read list (inbound continuation points).</param>
|
||||
/// <param name="results">The service-level results list.</param>
|
||||
/// <param name="errors">The service-level errors list.</param>
|
||||
/// <param name="startTimeUtc">Request StartTime.</param>
|
||||
/// <param name="endUtc">Request EndTime.</param>
|
||||
/// <param name="numValuesPerNode">Per-page cap; <c>0</c> = all.</param>
|
||||
/// <param name="ct">Request deadline.</param>
|
||||
private async Task ServeModifiedPagedAsync(
|
||||
NodeHandle handle,
|
||||
ISession? session,
|
||||
IList<HistoryReadValueId> nodesToRead,
|
||||
IList<SdkHistoryReadResult> results,
|
||||
IList<ServiceResult> errors,
|
||||
DateTime startTimeUtc,
|
||||
DateTime endUtc,
|
||||
uint numValuesPerNode,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var inboundCp = nodesToRead[handle.Index].ContinuationPoint;
|
||||
try
|
||||
{
|
||||
if (!TryGetHistorizedTagname(handle.NodeId, out var tagname) || tagname is null)
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||
return;
|
||||
}
|
||||
|
||||
var startUtc = startTimeUtc;
|
||||
HistoryModifiedCursor? resumeAfter = null;
|
||||
if (inboundCp is { Length: > 0 })
|
||||
{
|
||||
var state = _historyContinuationStore.TryTake(session, inboundCp);
|
||||
if (state is null || state.Kind != HistoryCursorKind.Modified || state.Tagname != tagname)
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadContinuationPointInvalid;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadContinuationPointInvalid };
|
||||
return;
|
||||
}
|
||||
|
||||
// The cursor carries the NORMALISED window, so a resumed page never re-derives its direction.
|
||||
startUtc = state.WindowStartUtc;
|
||||
endUtc = state.EndUtc;
|
||||
numValuesPerNode = state.NumValuesPerNode;
|
||||
resumeAfter = new HistoryModifiedCursor(state.NextStartUtc, state.LastModifiedId);
|
||||
}
|
||||
else if (numValuesPerNode > 0)
|
||||
{
|
||||
if (endUtc == DateTime.MinValue && startUtc != DateTime.MinValue)
|
||||
endUtc = new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc);
|
||||
else if (startUtc == DateTime.MinValue && endUtc != DateTime.MinValue)
|
||||
(startUtc, endUtc) = (endUtc, new DateTime(0, DateTimeKind.Utc));
|
||||
}
|
||||
|
||||
var page = await HistorianDataSource
|
||||
.ReadModifiedAsync(tagname, startUtc, endUtc, numValuesPerNode, resumeAfter, ct)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
var data = ToHistoryModifiedData(page.Values);
|
||||
byte[]? outboundCp = null;
|
||||
if (page.HasMore && numValuesPerNode > 0 && page.Values.Count > 0)
|
||||
{
|
||||
var last = page.Values[^1];
|
||||
outboundCp = _historyContinuationStore.Save(session, new HistoryContinuationState(
|
||||
tagname, last.Value.SourceTimestampUtc ?? DateTime.MinValue, endUtc, 0, numValuesPerNode,
|
||||
HistoryCursorKind.Modified, last.ModifiedId, startUtc));
|
||||
}
|
||||
|
||||
results[handle.Index] = new SdkHistoryReadResult
|
||||
{
|
||||
StatusCode = page.Values.Count == 0 ? StatusCodes.GoodNoData : StatusCodes.Good,
|
||||
HistoryData = new ExtensionObject(data),
|
||||
ContinuationPoint = outboundCp,
|
||||
};
|
||||
errors[handle.Index] = ServiceResult.Good;
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
errors[handle.Index] = StatusCodes.BadTimeout;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
||||
}
|
||||
catch (NotSupportedException ex)
|
||||
{
|
||||
// The backend keeps no modification trail (Gateway / Null) — the spec status, not a server fault.
|
||||
_logger.LogDebug(ex, "OtOpcUaNodeManager: backend serves no modified-value history for node {NodeId}", handle.NodeId);
|
||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "OtOpcUaNodeManager: HistoryRead (modified) failed for node {NodeId}", handle.NodeId);
|
||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Project a modification trail into an SDK <see cref="HistoryModifiedData"/>: one prior <see cref="DataValue"/>
|
||||
/// per modification plus a parallel <see cref="ModificationInfo"/> (time, Part 11 update type, user name).
|
||||
/// </summary>
|
||||
/// <param name="values">The modifications, in the order to emit.</param>
|
||||
/// <returns>The populated <see cref="HistoryModifiedData"/>.</returns>
|
||||
internal static HistoryModifiedData ToHistoryModifiedData(IReadOnlyList<HistoryModifiedValue> values)
|
||||
{
|
||||
var dataValues = new DataValueCollection(values.Count);
|
||||
var infos = new ModificationInfoCollection(values.Count);
|
||||
foreach (var m in values)
|
||||
{
|
||||
dataValues.Add(ToSdkDataValue(m.Value));
|
||||
infos.Add(new ModificationInfo
|
||||
{
|
||||
ModificationTime = m.ModificationTimeUtc,
|
||||
UpdateType = (HistoryUpdateType)(int)m.UpdateType,
|
||||
UserName = m.UserName,
|
||||
});
|
||||
}
|
||||
|
||||
return new HistoryModifiedData { DataValues = dataValues, ModificationInfos = infos };
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void HistoryReleaseContinuationPoints(
|
||||
ServerSystemContext context,
|
||||
|
||||
@@ -326,6 +326,20 @@ public sealed class FakeOpcUaClientService : IOpcUaClientService
|
||||
return Task.FromResult(HistoryReadResult);
|
||||
}
|
||||
|
||||
/// <summary>Gets the list of (NodeId, start, end, maxValues) tuples from HistoryReadModifiedAsync calls.</summary>
|
||||
public List<(NodeId NodeId, DateTime Start, DateTime End, int MaxValues)> HistoryReadModifiedCalls { get; } = [];
|
||||
|
||||
/// <summary>Gets or sets the entries returned by HistoryReadModifiedAsync.</summary>
|
||||
public IReadOnlyList<HistoryModifiedEntry> HistoryReadModifiedResult { get; set; } = [];
|
||||
|
||||
/// <inheritdoc />
|
||||
public Task<IReadOnlyList<HistoryModifiedEntry>> HistoryReadModifiedAsync(
|
||||
NodeId nodeId, DateTime startTime, DateTime endTime, int maxValues = 1000, CancellationToken ct = default)
|
||||
{
|
||||
HistoryReadModifiedCalls.Add((nodeId, startTime, endTime, maxValues));
|
||||
return Task.FromResult(HistoryReadModifiedResult);
|
||||
}
|
||||
|
||||
/// <summary>Gets the details passed to HistoryUpdateAsync.</summary>
|
||||
public List<HistoryUpdateDetails> HistoryUpdateCalls { get; } = [];
|
||||
|
||||
|
||||
@@ -146,4 +146,55 @@ public class HistoryReadCommandTests
|
||||
fakeService.DisconnectCalled.ShouldBeTrue();
|
||||
fakeService.DisposeCalled.ShouldBeTrue();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>#668: --modified reads the modification trail (not raw) and prints update type, user and prior value.</summary>
|
||||
[Fact]
|
||||
public async Task Execute_ModifiedRead_CallsHistoryReadModified_and_prints_the_trail()
|
||||
{
|
||||
var t = new DateTime(2025, 6, 15, 10, 0, 0, DateTimeKind.Utc);
|
||||
var fakeService = new FakeOpcUaClientService
|
||||
{
|
||||
HistoryReadModifiedResult =
|
||||
[
|
||||
new HistoryModifiedEntry(new DataValue(Variant.Null, StatusCodes.BadNoData, t, t), t.AddDays(1), HistoryUpdateType.Insert, "editor"),
|
||||
new HistoryModifiedEntry(new DataValue(new Variant(1.5), StatusCodes.Good, t, t), t.AddDays(2), HistoryUpdateType.Replace, "editor"),
|
||||
],
|
||||
};
|
||||
var command = new HistoryReadCommand(new FakeOpcUaClientServiceFactory(fakeService))
|
||||
{
|
||||
Url = "opc.tcp://localhost:4840",
|
||||
NodeId = "ns=2;s=HistNode",
|
||||
Modified = true,
|
||||
MaxValues = 50,
|
||||
};
|
||||
|
||||
using var console = TestConsoleHelper.CreateConsole();
|
||||
await command.ExecuteAsync(console);
|
||||
|
||||
fakeService.HistoryReadModifiedCalls.ShouldHaveSingleItem().MaxValues.ShouldBe(50);
|
||||
fakeService.HistoryReadRawCalls.ShouldBeEmpty();
|
||||
var output = TestConsoleHelper.GetOutput(console);
|
||||
output.ShouldContain("Modified history for ns=2;s=HistNode");
|
||||
output.ShouldContain("Replace");
|
||||
output.ShouldContain("Insert");
|
||||
output.ShouldContain("editor");
|
||||
output.ShouldContain("(none)");
|
||||
output.ShouldContain("2 modifications returned.");
|
||||
}
|
||||
|
||||
/// <summary>--modified cannot be combined with --aggregate.</summary>
|
||||
[Fact]
|
||||
public async Task Execute_ModifiedWithAggregate_ThrowsCommandException()
|
||||
{
|
||||
var command = new HistoryReadCommand(new FakeOpcUaClientServiceFactory(new FakeOpcUaClientService()))
|
||||
{
|
||||
Url = "opc.tcp://localhost:4840",
|
||||
NodeId = "ns=2;s=HistNode",
|
||||
Modified = true,
|
||||
Aggregate = "Average",
|
||||
};
|
||||
|
||||
using var console = TestConsoleHelper.CreateConsole();
|
||||
await Should.ThrowAsync<CliFx.Exceptions.CommandException>(async () => await command.ExecuteAsync(console));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,6 +8,9 @@ public static class ConformanceData
|
||||
/// <summary>Base time: 2026-10-09 08:00:00.1234567 UTC (sub-millisecond ticks on purpose).</summary>
|
||||
public static readonly DateTime T0 = new DateTime(2026, 10, 9, 8, 0, 0, DateTimeKind.Utc).AddTicks(1_234_567);
|
||||
|
||||
/// <summary>The HistoryUpdate caller the suite passes to every updater call (recorded in the modification trail).</summary>
|
||||
public static readonly ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryUpdateContext Editor = new("conformance-editor", "k-conf");
|
||||
|
||||
/// <summary>Good.</summary>
|
||||
public const uint Good = 0x00000000u;
|
||||
/// <summary>Bad (generic).</summary>
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;
|
||||
|
||||
/// <summary>
|
||||
/// OPC UA Part 11 modified-value history (Gitea #668): every HistoryUpdate a backend applies leaves an ordered
|
||||
/// modification trail — the PRIOR value, the caller's principal, the Part 11 update type and the modification time —
|
||||
/// readable through <see cref="IHistorianDataSource.ReadModifiedAsync"/>. Refused items, live recorder writes and
|
||||
/// reads leave nothing.
|
||||
/// </summary>
|
||||
public abstract class ModifiedValueConformance : ConformanceBase
|
||||
{
|
||||
private static readonly DateTime T0 = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
|
||||
private static DateTime T(int n) => T0.AddSeconds(n).AddTicks(1230); // µs-aligned so both backends store it exactly
|
||||
private static readonly DateTime Lo = T0.AddHours(-1);
|
||||
private static readonly DateTime Hi = T0.AddHours(1);
|
||||
private static readonly CancellationToken Ct = CancellationToken.None;
|
||||
private static readonly HistoryUpdateContext Editor = ConformanceData.Editor;
|
||||
|
||||
private static HistorizationValue V(int n, double d, uint status = Good) =>
|
||||
new(T(n), HistorianValue.FromDouble(d), status);
|
||||
|
||||
private async Task<IReadOnlyList<HistoryModifiedValue>> TrailAsync(
|
||||
string tag, DateTime? start = null, DateTime? end = null, uint max = 0, HistoryModifiedCursor? after = null)
|
||||
{
|
||||
using var src = B.CreateDataSource();
|
||||
var r = await src.ReadModifiedAsync(tag, start ?? Lo, end ?? Hi, max, after, Ct);
|
||||
if (max == 0) r.HasMore.ShouldBeFalse();
|
||||
return r.Values;
|
||||
}
|
||||
|
||||
private static void ShouldBeNoPrior(HistoryModifiedValue m)
|
||||
{
|
||||
m.Value.Value.ShouldBeNull();
|
||||
m.Value.StatusCode.ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Insert_records_one_Insert_per_sample_with_no_prior_value_and_the_principal()
|
||||
{
|
||||
var before = DateTime.UtcNow.AddSeconds(-5);
|
||||
var u = B.CreateUpdater();
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(2, 2), V(1, 1)], Editor, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryInserted]);
|
||||
|
||||
var trail = await TrailAsync("t");
|
||||
|
||||
trail.Count.ShouldBe(2);
|
||||
trail.Select(m => m.Value.SourceTimestampUtc).ShouldBe([T(1), T(2)]); // ordered by source time
|
||||
trail.ShouldAllBe(m => m.UpdateType == HistoryModificationType.Insert);
|
||||
trail.ShouldAllBe(m => m.UserName == "conformance-editor");
|
||||
trail.ShouldAllBe(m => m.ModificationTimeUtc >= before && m.ModificationTimeUtc <= DateTime.UtcNow.AddSeconds(5));
|
||||
foreach (var m in trail) ShouldBeNoPrior(m);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Replace_records_the_prior_value_and_status_after_the_insert()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1)], Editor, Ct);
|
||||
var other = new HistoryUpdateContext("second-editor", null);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(1, 11, Uncertain)], other, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryReplaced]);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(1, 111)], Editor, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryReplaced]);
|
||||
|
||||
var trail = await TrailAsync("t");
|
||||
|
||||
trail.Select(m => m.UpdateType).ShouldBe(
|
||||
[HistoryModificationType.Insert, HistoryModificationType.Replace, HistoryModificationType.Replace]);
|
||||
trail.Select(m => m.UserName).ShouldBe(["conformance-editor", "second-editor", "conformance-editor"]);
|
||||
ShouldBeNoPrior(trail[0]);
|
||||
trail[1].Value.Value.ShouldBe(1d); // what the first Replace overwrote
|
||||
trail[1].Value.StatusCode.ShouldBe(Good);
|
||||
trail[2].Value.Value.ShouldBe(11d); // what the second Replace overwrote, with its status
|
||||
trail[2].Value.StatusCode.ShouldBe(Uncertain);
|
||||
trail.ShouldAllBe(m => m.Value.SourceTimestampUtc == T(1));
|
||||
trail.Select(m => m.ModifiedId).ShouldBeInOrder(SortDirection.Ascending);
|
||||
trail.Select(m => m.ModifiedId).Distinct().Count().ShouldBe(3);
|
||||
// The sample itself carries the latest value.
|
||||
(await B.ReadBackAsync("t")).ShouldHaveSingleItem().V.Number.ShouldBe(111d);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Update_records_Update_with_the_prior_value_when_it_replaced_and_none_when_it_inserted()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1)], Editor, Ct);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Update, [V(2, 2), V(1, 10)], Editor, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryReplaced]);
|
||||
|
||||
var trail = await TrailAsync("t");
|
||||
|
||||
trail.Select(m => (m.Value.SourceTimestampUtc, m.UpdateType)).ShouldBe(
|
||||
[
|
||||
(T(1), HistoryModificationType.Insert),
|
||||
(T(1), HistoryModificationType.Update),
|
||||
(T(2), HistoryModificationType.Update),
|
||||
]);
|
||||
trail[1].Value.Value.ShouldBe(1d);
|
||||
ShouldBeNoPrior(trail[2]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DeleteAtTime_records_a_Delete_carrying_each_deleted_value()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1), V(2, 2), V(3, 3, Uncertain)], Editor, Ct);
|
||||
var deleter = new HistoryUpdateContext("deleter", "k-del");
|
||||
(await u.DeleteAtTimeAsync("t", [T(3), T(1), T(9)], deleter, Ct))
|
||||
.ShouldBe([Good, Good, HistorianStatusCodes.BadNoEntryExists]);
|
||||
|
||||
var deletes = (await TrailAsync("t")).Where(m => m.UpdateType == HistoryModificationType.Delete).ToList();
|
||||
|
||||
deletes.Select(m => m.Value.SourceTimestampUtc).ShouldBe([T(1), T(3)]);
|
||||
deletes.Select(m => (double)m.Value.Value!).ShouldBe([1d, 3d]);
|
||||
deletes[1].Value.StatusCode.ShouldBe(Uncertain);
|
||||
deletes.ShouldAllBe(m => m.UserName == "deleter");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task DeleteRaw_records_a_Delete_for_every_sample_it_removed()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1), V(2, 2), V(3, 3)], Editor, Ct);
|
||||
(await u.DeleteRawAsync("t", T(1), T(3), Editor, Ct)).ShouldBe(Good); // [T1, T3)
|
||||
|
||||
var deletes = (await TrailAsync("t")).Where(m => m.UpdateType == HistoryModificationType.Delete).ToList();
|
||||
|
||||
deletes.Select(m => m.Value.SourceTimestampUtc).ShouldBe([T(1), T(2)]);
|
||||
deletes.Select(m => (double)m.Value.Value!).ShouldBe([1d, 2d]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Refused_items_leave_no_trail()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1)], Editor, Ct);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 99)], Editor, Ct)).ShouldBe([HistorianStatusCodes.BadEntryExists]);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(5, 5)], Editor, Ct)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [new HistorizationValue(null, HistorianValue.FromDouble(1), Good)], Editor, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
(await u.DeleteAtTimeAsync("t", [T(7)], Editor, Ct)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await u.DeleteRawAsync("t", T(10), T(20), Editor, Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
|
||||
var trail = await TrailAsync("t");
|
||||
trail.ShouldHaveSingleItem().UpdateType.ShouldBe(HistoryModificationType.Insert); // only the first, applied insert
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Live_recorder_writes_leave_no_trail()
|
||||
{
|
||||
var w = B.CreateValueWriter();
|
||||
(await w.WriteLiveValuesAsync("live", [new(T(1), HistorianValue.FromDouble(1), Good), new(T(2), HistorianValue.FromDouble(2), Good)], Ct))
|
||||
.ShouldBeTrue();
|
||||
(await B.ReadBackAsync("live")).Count.ShouldBe(2); // the writes landed …
|
||||
(await TrailAsync("live")).ShouldBeEmpty(); // … and recorded no modification
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_unknown_tag_has_an_empty_trail_and_reads_create_no_tag()
|
||||
{
|
||||
(await TrailAsync("never")).ShouldBeEmpty();
|
||||
(await B.TagExistsAsync("never")).ShouldBeFalse();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task The_trail_pages_by_key_exactly_once_and_reports_HasMore()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1), V(2, 2), V(3, 3)], Editor, Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(2, 20)], Editor, Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(2, 200)], Editor, Ct);
|
||||
var all = await TrailAsync("t");
|
||||
all.Count.ShouldBe(5); // T1 ins | T2 ins, rep, rep | T3 ins
|
||||
|
||||
var seen = new List<long>();
|
||||
HistoryModifiedCursor? cursor = null;
|
||||
using var src = B.CreateDataSource();
|
||||
for (var page = 0; page < 10; page++)
|
||||
{
|
||||
var r = await src.ReadModifiedAsync("t", Lo, Hi, 2, cursor, Ct);
|
||||
seen.AddRange(r.Values.Select(m => m.ModifiedId));
|
||||
if (!r.HasMore) break;
|
||||
r.Values.Count.ShouldBe(2);
|
||||
var last = r.Values[^1];
|
||||
cursor = new HistoryModifiedCursor(last.Value.SourceTimestampUtc!.Value, last.ModifiedId);
|
||||
}
|
||||
|
||||
seen.ShouldBe(all.Select(m => m.ModifiedId)); // a page boundary inside the T2 run neither repeats nor skips
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_reverse_window_reads_the_trail_descending()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1), V(2, 2)], Editor, Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(2, 20)], Editor, Ct);
|
||||
var forward = await TrailAsync("t");
|
||||
|
||||
var reverse = await TrailAsync("t", start: Hi, end: Lo);
|
||||
|
||||
reverse.Select(m => m.ModifiedId).ShouldBe(forward.Select(m => m.ModifiedId).Reverse());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task The_window_is_half_open_on_source_time()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(1, 1), V(2, 2), V(3, 3)], Editor, Ct);
|
||||
(await TrailAsync("t", start: T(1), end: T(3))).Select(m => m.Value.SourceTimestampUtc).ShouldBe([T(1), T(2)]);
|
||||
(await TrailAsync("t", start: T(2), end: T(2))).Select(m => m.Value.SourceTimestampUtc).ShouldBe([T(2)]);
|
||||
}
|
||||
}
|
||||
@@ -85,7 +85,7 @@ public abstract class QualityContractConformance : ConformanceBase
|
||||
new(At(1), HistorianValue.Empty, 0x40000000u),
|
||||
new(At(2), HistorianValue.Empty, BadNoCommunication),
|
||||
new(At(3), HistorianValue.FromDouble(1), 0u),
|
||||
], Ct);
|
||||
], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([BadInvalidArgument, BadInvalidArgument, HistorianStatusCodes.GoodEntryInserted,
|
||||
HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await B.CreateDataSource().ReadRawAsync("u", At(0), At(10), 0, Ct)).Samples
|
||||
|
||||
@@ -39,10 +39,10 @@ public abstract class TagNameConformance : ConformanceBase
|
||||
var u = B.CreateUpdater();
|
||||
|
||||
(await u.UpdateDataAsync(Long, HistoryUpdateMode.Update,
|
||||
[new(At(0), HistorianValue.FromDouble(9), 0u), new(At(1), HistorianValue.FromDouble(9), 0u)], Ct))
|
||||
[new(At(0), HistorianValue.FromDouble(9), 0u), new(At(1), HistorianValue.FromDouble(9), 0u)], ConformanceData.Editor, Ct))
|
||||
.ShouldBe([BadInvalidArgument, BadInvalidArgument]);
|
||||
(await u.DeleteRawAsync(Long, At(-1), At(10), Ct)).ShouldBe(BadInvalidArgument);
|
||||
(await u.DeleteAtTimeAsync(Long, [At(0)], Ct)).ShouldBe([BadInvalidArgument]);
|
||||
(await u.DeleteRawAsync(Long, At(-1), At(10), ConformanceData.Editor, Ct)).ShouldBe(BadInvalidArgument);
|
||||
(await u.DeleteAtTimeAsync(Long, [At(0)], ConformanceData.Editor, Ct)).ShouldBe([BadInvalidArgument]);
|
||||
|
||||
// The prefix series is untouched: not replaced, not deleted.
|
||||
var raw = (await B.CreateDataSource().ReadRawAsync(Prefix, At(-1), At(10), 0, Ct)).Samples;
|
||||
|
||||
@@ -23,10 +23,10 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Insert_new_is_GoodEntryInserted_and_existing_is_BadEntryExists_in_order()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
var r1 = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2)], Ct);
|
||||
var r1 = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2)], ConformanceData.Editor, Ct);
|
||||
r1.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryInserted]);
|
||||
// Mixed: T(2) exists, T(3) new, T(1) exists - out of timestamp order on purpose.
|
||||
var r2 = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(3), 3), V(T(2), 99), V(T(1), 98)], Ct);
|
||||
var r2 = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(3), 3), V(T(2), 99), V(T(1), 98)], ConformanceData.Editor, Ct);
|
||||
r2.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.BadEntryExists, HistorianStatusCodes.BadEntryExists]);
|
||||
var rows = await B.ReadBackAsync("t");
|
||||
rows.Select(x => (double?)x.V.Number).ShouldBe([1d, 2d, 3d]); // existing values untouched
|
||||
@@ -37,9 +37,9 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Replace_existing_is_GoodEntryReplaced_with_new_value_and_status_missing_is_BadNoEntryExists()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2)], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2)], ConformanceData.Editor, Ct);
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Replace,
|
||||
[V(T(2), 22, UncertainStatus), V(T(5), 5), V(T(1), 11)], Ct);
|
||||
[V(T(2), 22, UncertainStatus), V(T(5), 5), V(T(1), 11)], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.BadNoEntryExists, HistorianStatusCodes.GoodEntryReplaced]);
|
||||
var rows = await B.ReadBackAsync("t");
|
||||
rows.Count.ShouldBe(2); // T(5) was not created
|
||||
@@ -53,8 +53,8 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Update_upserts_inserted_or_replaced_per_item()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], Ct);
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Update, [V(T(2), 2), V(T(1), 10)], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], ConformanceData.Editor, Ct);
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Update, [V(T(2), 2), V(T(1), 10)], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryReplaced]);
|
||||
(await B.ReadBackAsync("t")).Select(x => (double?)x.V.Number).ShouldBe([10d, 2d]);
|
||||
}
|
||||
@@ -63,15 +63,15 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Unknown_tag_Insert_and_Update_create_it_but_Replace_and_deletes_do_not()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
(await u.UpdateDataAsync("rep", HistoryUpdateMode.Replace, [V(T(1), 1)], Ct))
|
||||
(await u.UpdateDataAsync("rep", HistoryUpdateMode.Replace, [V(T(1), 1)], ConformanceData.Editor, Ct))
|
||||
.ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await u.DeleteRawAsync("del", T(0), T(9), Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await u.DeleteAtTimeAsync("del2", [T(1)], Ct)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await u.DeleteRawAsync("del", T(0), T(9), ConformanceData.Editor, Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await u.DeleteAtTimeAsync("del2", [T(1)], ConformanceData.Editor, Ct)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
foreach (var name in new[] { "rep", "del", "del2" })
|
||||
(await B.TagExistsAsync(name)).ShouldBeFalse(name); // none of those created a tag
|
||||
|
||||
(await u.UpdateDataAsync("ins", HistoryUpdateMode.Insert, [V(T(1), 1)], Ct)).ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await u.UpdateDataAsync("upd", HistoryUpdateMode.Update, [V(T(1), 1)], Ct)).ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await u.UpdateDataAsync("ins", HistoryUpdateMode.Insert, [V(T(1), 1)], ConformanceData.Editor, Ct)).ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await u.UpdateDataAsync("upd", HistoryUpdateMode.Update, [V(T(1), 1)], ConformanceData.Editor, Ct)).ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await B.TagExistsAsync("ins")).ShouldBeTrue();
|
||||
(await B.TagExistsAsync("upd")).ShouldBeTrue();
|
||||
}
|
||||
@@ -81,14 +81,14 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert,
|
||||
[V(null, 1), V(T(1), 2), V(DateTime.MinValue, 3)], Ct);
|
||||
[V(null, 1), V(T(1), 2), V(DateTime.MinValue, 3)], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([HistorianStatusCodes.BadInvalidTimestamp, HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
(await B.ReadBackAsync("t")).Count.ShouldBe(1);
|
||||
|
||||
// All-invalid request: no DB work, no tag created, still one code per item.
|
||||
var u2 = B.CreateUpdater();
|
||||
await u2.UpdateDataAsync("seed", HistoryUpdateMode.Insert, [V(T(1), 1)], Ct); // bootstraps the schema
|
||||
(await u2.UpdateDataAsync("x", HistoryUpdateMode.Insert, [V(null, 1)], Ct)).ShouldBe([HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
await u2.UpdateDataAsync("seed", HistoryUpdateMode.Insert, [V(T(1), 1)], ConformanceData.Editor, Ct); // bootstraps the schema
|
||||
(await u2.UpdateDataAsync("x", HistoryUpdateMode.Insert, [V(null, 1)], ConformanceData.Editor, Ct)).ShouldBe([HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
(await B.TagExistsAsync("seed")).ShouldBeTrue();
|
||||
(await B.TagExistsAsync("x")).ShouldBeFalse(); // "x" was never created
|
||||
}
|
||||
@@ -98,7 +98,7 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert,
|
||||
[V(T(1), double.NaN), V(T(2), 2), V(T(3), double.PositiveInfinity), V(T(4), double.NegativeInfinity)], Ct);
|
||||
[V(T(1), double.NaN), V(T(2), 2), V(T(3), double.PositiveInfinity), V(T(4), double.NegativeInfinity)], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([HistorianStatusCodes.BadInvalidArgument, HistorianStatusCodes.GoodEntryInserted,
|
||||
HistorianStatusCodes.BadInvalidArgument, HistorianStatusCodes.BadInvalidArgument]);
|
||||
(await B.ReadBackAsync("t")).Select(x => (double?)x.V.Number).ShouldBe([2d]);
|
||||
@@ -108,7 +108,7 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Duplicate_timestamp_in_one_Insert_request_second_is_BadEntryExists_first_wins()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(1), 99)], Ct);
|
||||
var r = await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(1), 99)], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.BadEntryExists]);
|
||||
(await B.ReadBackAsync("t")).Select(x => (double?)x.V.Number).ShouldBe([1d, 2d]);
|
||||
}
|
||||
@@ -117,12 +117,12 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Duplicate_timestamp_in_one_Replace_or_Update_request_last_wins_and_all_share_the_code()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], Ct);
|
||||
var rep = await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(T(1), 5), V(T(1), 6)], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], ConformanceData.Editor, Ct);
|
||||
var rep = await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(T(1), 5), V(T(1), 6)], ConformanceData.Editor, Ct);
|
||||
rep.ShouldBe([HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.GoodEntryReplaced]);
|
||||
(await B.ReadBackAsync("t")).Single().V.Number.ShouldBe(6d);
|
||||
|
||||
var upd = await u.UpdateDataAsync("t", HistoryUpdateMode.Update, [V(T(7), 7), V(T(7), 8)], Ct);
|
||||
var upd = await u.UpdateDataAsync("t", HistoryUpdateMode.Update, [V(T(7), 7), V(T(7), 8)], ConformanceData.Editor, Ct);
|
||||
upd.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await B.ReadBackAsync("t")).Last().V.Number.ShouldBe(8d);
|
||||
}
|
||||
@@ -131,9 +131,9 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task Replace_and_Update_can_change_the_value_kind_and_clear_stale_columns()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [new HistorizationValue(T(1), HistorianValue.FromDouble(1.5), Good)], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [new HistorizationValue(T(1), HistorianValue.FromDouble(1.5), Good)], ConformanceData.Editor, Ct);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Replace,
|
||||
[new HistorizationValue(T(1), HistorianValue.FromString("hi"), Good)], Ct)).ShouldBe([HistorianStatusCodes.GoodEntryReplaced]);
|
||||
[new HistorizationValue(T(1), HistorianValue.FromString("hi"), Good)], ConformanceData.Editor, Ct)).ShouldBe([HistorianStatusCodes.GoodEntryReplaced]);
|
||||
var rows = await B.ReadBackAsync("t");
|
||||
rows.Single().V.ShouldBe(HistorianValue.FromString("hi")); // Number cleared (0), not left at 1.5
|
||||
}
|
||||
@@ -142,13 +142,13 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task DeleteRaw_is_half_open_Good_when_rows_removed_BadNoData_when_none()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(3), 3)], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(3), 3)], ConformanceData.Editor, Ct);
|
||||
// [T(1), T(3)) removes T(1) and T(2); T(3) (the end) survives.
|
||||
(await u.DeleteRawAsync("t", T(1), T(3), Ct)).ShouldBe(Good);
|
||||
(await u.DeleteRawAsync("t", T(1), T(3), ConformanceData.Editor, Ct)).ShouldBe(Good);
|
||||
(await B.ReadBackAsync("t")).Select(x => x.T).ShouldBe([B.Round(T(3))]);
|
||||
(await u.DeleteRawAsync("t", T(1), T(3), Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await u.DeleteRawAsync("t", T(1), T(3), ConformanceData.Editor, Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
// sub-ms boundary: a start one resolution unit after T(3) excludes it.
|
||||
(await u.DeleteRawAsync("t", T(3).AddTicks(B.ResolutionTicks), T(3).AddTicks(100 * B.ResolutionTicks), Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await u.DeleteRawAsync("t", T(3).AddTicks(B.ResolutionTicks), T(3).AddTicks(100 * B.ResolutionTicks), ConformanceData.Editor, Ct)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await B.ReadBackAsync("t")).Count.ShouldBe(1);
|
||||
}
|
||||
|
||||
@@ -156,8 +156,8 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task DeleteAtTime_reports_per_time_in_order_with_exact_tick_matching()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(3), 3)], Ct);
|
||||
var r = await u.DeleteAtTimeAsync("t", [T(3), T(2).AddTicks(B.ResolutionTicks) /* near miss */, T(1), DateTime.MinValue], Ct);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1), V(T(2), 2), V(T(3), 3)], ConformanceData.Editor, Ct);
|
||||
var r = await u.DeleteAtTimeAsync("t", [T(3), T(2).AddTicks(B.ResolutionTicks) /* near miss */, T(1), DateTime.MinValue], ConformanceData.Editor, Ct);
|
||||
r.ShouldBe([Good, HistorianStatusCodes.BadNoEntryExists, Good, HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
(await B.ReadBackAsync("t")).Select(x => x.T).ShouldBe([B.Round(T(2))]);
|
||||
}
|
||||
@@ -166,8 +166,8 @@ public abstract class UpdateConformance : ConformanceBase
|
||||
public async Task DeleteAtTime_duplicate_time_first_reports_Good_later_BadNoEntryExists()
|
||||
{
|
||||
var u = B.CreateUpdater();
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], Ct);
|
||||
(await u.DeleteAtTimeAsync("t", [T(1), T(1)], Ct)).ShouldBe([Good, HistorianStatusCodes.BadNoEntryExists]);
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T(1), 1)], ConformanceData.Editor, Ct);
|
||||
(await u.DeleteAtTimeAsync("t", [T(1), T(1)], ConformanceData.Editor, Ct)).ShouldBe([Good, HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await B.ReadBackAsync("t")).ShouldBeEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
+5
-3
@@ -52,7 +52,7 @@ public sealed class BootstrapTests(QuestDbFixture fx)
|
||||
var p = fx.UniquePrefix();
|
||||
var (c, boot, _) = Make(p);
|
||||
await boot.EnsureAsync(default);
|
||||
(await TablesAsync(c, p)).ShouldBe([p + "event", p + "sample", p + "tag"]);
|
||||
(await TablesAsync(c, p)).ShouldBe([p + "event", p + "sample", p + "sample_modified", p + "tag"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -69,7 +69,7 @@ public sealed class BootstrapTests(QuestDbFixture fx)
|
||||
await Task.Delay(300); // an ALTER would have been applied by now
|
||||
(await SequencerTxnAsync(c2, p + "sample")).ShouldBe(before);
|
||||
(await SequencerTxnAsync(c2, p + "event")).ShouldBe(await SequencerTxnAsync(c, p + "event"));
|
||||
(await TablesAsync(c2, p)).Count.ShouldBe(3);
|
||||
(await TablesAsync(c2, p)).Count.ShouldBe(4);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -84,7 +84,7 @@ public sealed class BootstrapTests(QuestDbFixture fx)
|
||||
await Task.WhenAll(
|
||||
Task.Run(() => a.Boot.EnsureAsync(default)),
|
||||
Task.Run(() => b.Boot.EnsureAsync(default)));
|
||||
(await TablesAsync(a.Conn, p)).Count.ShouldBe(3);
|
||||
(await TablesAsync(a.Conn, p)).Count.ShouldBe(4);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -97,6 +97,8 @@ public sealed class BootstrapTests(QuestDbFixture fx)
|
||||
await b1.EnsureAsync(default);
|
||||
QuestDbTables.TtlHours((await TtlAsync(c1, withTtl + "sample")).Value, (await TtlAsync(c1, withTtl + "sample")).Unit).ShouldBe(168);
|
||||
QuestDbTables.TtlHours((await TtlAsync(c1, withTtl + "event")).Value, (await TtlAsync(c1, withTtl + "event")).Unit).ShouldBe(168);
|
||||
// #668: the modification trail ages out with the samples it describes.
|
||||
QuestDbTables.TtlHours((await TtlAsync(c1, withTtl + "sample_modified")).Value, (await TtlAsync(c1, withTtl + "sample_modified")).Unit).ShouldBe(168);
|
||||
(await TtlAsync(c1, withTtl + "tag")).Value.ShouldBe(0); // the tag catalog never expires
|
||||
|
||||
var noTtl = fx.UniquePrefix();
|
||||
|
||||
+7
@@ -53,3 +53,10 @@ public sealed class QuestDbTagNameConformance(QuestDbFixture fx) : TagNameConfor
|
||||
/// <inheritdoc/>
|
||||
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
|
||||
}
|
||||
|
||||
[Collection(QuestDbCollection.Name)]
|
||||
public sealed class QuestDbModifiedValueConformance(QuestDbFixture fx) : ModifiedValueConformance
|
||||
{
|
||||
/// <inheritdoc/>
|
||||
protected override Task<IConformanceBackend> CreateBackendAsync() => QuestDbConformanceBackend.CreateAsync(fx);
|
||||
}
|
||||
|
||||
+1
-1
@@ -26,7 +26,7 @@ public sealed partial class TombstoneTests
|
||||
];
|
||||
|
||||
(await writer.WriteLiveValuesAsync("r", batch, default)).ShouldBeTrue();
|
||||
(await rig.Updater.DeleteAtTimeAsync("r", [T0], default)).ShouldBe([HistorianStatusCodes.Good]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("r", [T0], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.Good]);
|
||||
(await writer.WriteLiveValuesAsync("r", batch, default)).ShouldBeTrue(); // the redelivery
|
||||
|
||||
(await UpdateTests.RowsAsync(rig, "r", liveOnly: true)).Select(r => r.Num).ShouldBe([2d]);
|
||||
|
||||
+6
-6
@@ -17,8 +17,8 @@ public sealed partial class TombstoneTests(QuestDbFixture fx)
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = UpdateTests.MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert,
|
||||
[UpdateTests.V(T0, 1), UpdateTests.V(T0.AddSeconds(1), 2)], default);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0], default)).ShouldBe([HistorianStatusCodes.Good]);
|
||||
[UpdateTests.V(T0, 1), UpdateTests.V(T0.AddSeconds(1), 2)], UpdateTests.Who, default);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.Good]);
|
||||
|
||||
(await UpdateTests.RowsAsync(rig, "t", liveOnly: true)).Select(r => r.Num).ShouldBe([2d]);
|
||||
var all = await UpdateTests.RowsAsync(rig, "t");
|
||||
@@ -35,12 +35,12 @@ public sealed partial class TombstoneTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = UpdateTests.MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [UpdateTests.V(T0, 1)], default);
|
||||
await rig.Updater.DeleteAtTimeAsync("t", [T0], default);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [UpdateTests.V(T0, 1)], UpdateTests.Who, default);
|
||||
await rig.Updater.DeleteAtTimeAsync("t", [T0], UpdateTests.Who, default);
|
||||
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Replace, [UpdateTests.V(T0, 5)], default))
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Replace, [UpdateTests.V(T0, 5)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.BadNoEntryExists]); // a tombstone is no entry for Replace
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [UpdateTests.V(T0, 9)], default))
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [UpdateTests.V(T0, 9)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
var live = await UpdateTests.RowsAsync(rig, "t", liveOnly: true);
|
||||
live.Count.ShouldBe(1);
|
||||
|
||||
+37
-34
@@ -24,6 +24,9 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
return new Rig(c, new QuestDbHistorianDataUpdater(c, tags, wal), tags);
|
||||
}
|
||||
|
||||
/// <summary>The HistoryUpdate caller these tests pass to the updater (recorded in the modification trail).</summary>
|
||||
internal static readonly HistoryUpdateContext Who = new("questdb-test", null);
|
||||
|
||||
internal static HistorizationValue V(DateTime? t, double v, uint status = Good) =>
|
||||
new(t, HistorianValue.FromDouble(v), status);
|
||||
|
||||
@@ -57,10 +60,10 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
(await rig.Updater.UpdateDataAsync("t1", HistoryUpdateMode.Insert, [V(T0, 1)], default))
|
||||
(await rig.Updater.UpdateDataAsync("t1", HistoryUpdateMode.Insert, [V(T0, 1)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
var codes = await rig.Updater.UpdateDataAsync("t1", HistoryUpdateMode.Insert,
|
||||
[V(T0, 99), V(T0.AddSeconds(1), 2)], default);
|
||||
[V(T0, 99), V(T0.AddSeconds(1), 2)], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.BadEntryExists, HistorianStatusCodes.GoodEntryInserted]);
|
||||
var rows = await RowsAsync(rig, "t1"); // read right after return: read-your-writes
|
||||
rows.Select(r => r.Num).ShouldBe([1d, 2d]);
|
||||
@@ -72,14 +75,14 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
(await rig.Updater.UpdateDataAsync("nope", HistoryUpdateMode.Replace, [V(T0, 1)], default))
|
||||
(await rig.Updater.UpdateDataAsync("nope", HistoryUpdateMode.Replace, [V(T0, 1)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await TagRowsAsync(rig, "nope")).ShouldBe(0);
|
||||
(await RowsAsync(rig, "nope")).ShouldBeEmpty();
|
||||
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], default);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], UpdateTests.Who, default);
|
||||
var codes = await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Replace,
|
||||
[V(T0, 5), V(T0.AddSeconds(1), 6)], default);
|
||||
[V(T0, 5), V(T0.AddSeconds(1), 6)], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await RowsAsync(rig, "t")).Select(r => r.Num).ShouldBe([5d]);
|
||||
}
|
||||
@@ -89,9 +92,9 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], default);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], UpdateTests.Who, default);
|
||||
var codes = await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Update,
|
||||
[V(T0, 2), V(T0.AddSeconds(1), 3), V(T0.AddSeconds(1).AddTicks(3), 4)], default);
|
||||
[V(T0, 2), V(T0.AddSeconds(1), 3), V(T0.AddSeconds(1).AddTicks(3), 4)], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.GoodEntryInserted,
|
||||
HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await RowsAsync(rig, "t")).Select(r => r.Num).ShouldBe([2d, 4d]);
|
||||
@@ -104,7 +107,7 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
var rig = MakeRig(fx);
|
||||
// offsets 0 and 3 ticks sit inside one 10-tick window => the same stored µs
|
||||
var codes = await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert,
|
||||
[V(T0, 1), V(T0.AddTicks(3), 2)], default);
|
||||
[V(T0, 1), V(T0.AddTicks(3), 2)], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.BadEntryExists]);
|
||||
(await RowsAsync(rig, "t")).Select(r => r.Num).ShouldBe([1d]);
|
||||
}
|
||||
@@ -119,7 +122,7 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
V(null, 1), V(DateTime.MinValue, 1), V(T0, double.NaN),
|
||||
new HistorizationValue(T0.AddSeconds(1), HistorianValue.Empty, Good),
|
||||
V(T0.AddSeconds(2), 7),
|
||||
], default);
|
||||
], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.BadInvalidTimestamp, HistorianStatusCodes.BadInvalidTimestamp,
|
||||
HistorianStatusCodes.BadInvalidArgument, HistorianStatusCodes.BadInvalidArgument,
|
||||
HistorianStatusCodes.GoodEntryInserted]);
|
||||
@@ -132,10 +135,10 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
var tag = new string('x', HistorianTagNames.MaxLength + 1);
|
||||
(await rig.Updater.UpdateDataAsync(tag, HistoryUpdateMode.Insert, [V(T0, 1)], default))
|
||||
(await rig.Updater.UpdateDataAsync(tag, HistoryUpdateMode.Insert, [V(T0, 1)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.BadInvalidArgument]);
|
||||
(await rig.Updater.DeleteAtTimeAsync(tag, [T0], default)).ShouldBe([HistorianStatusCodes.BadInvalidArgument]);
|
||||
(await rig.Updater.DeleteRawAsync(tag, T0, T0.AddHours(1), default)).ShouldBe(HistorianStatusCodes.BadInvalidArgument);
|
||||
(await rig.Updater.DeleteAtTimeAsync(tag, [T0], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.BadInvalidArgument]);
|
||||
(await rig.Updater.DeleteRawAsync(tag, T0, T0.AddHours(1), UpdateTests.Who, default)).ShouldBe(HistorianStatusCodes.BadInvalidArgument);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -146,15 +149,15 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
var pre = new DateTime(1969, 12, 31, 23, 59, 59, DateTimeKind.Utc);
|
||||
foreach (var mode in new[] { HistoryUpdateMode.Insert, HistoryUpdateMode.Replace, HistoryUpdateMode.Update })
|
||||
{
|
||||
var codes = await rig.Updater.UpdateDataAsync("t", mode, [V(pre, 1), V(DateTime.MinValue, 1), V(T0, 2)], default);
|
||||
var codes = await rig.Updater.UpdateDataAsync("t", mode, [V(pre, 1), V(DateTime.MinValue, 1), V(T0, 2)], UpdateTests.Who, default);
|
||||
codes[0].ShouldBe(HistorianStatusCodes.BadOutOfRange);
|
||||
codes[1].ShouldBe(HistorianStatusCodes.BadInvalidTimestamp);
|
||||
}
|
||||
// Pre-1970 only: the whole batch is refused, no tag row is created, no INSERT would succeed anyway.
|
||||
(await rig.Updater.UpdateDataAsync("only-pre", HistoryUpdateMode.Update, [V(pre, 1)], default))
|
||||
(await rig.Updater.UpdateDataAsync("only-pre", HistoryUpdateMode.Update, [V(pre, 1)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.BadOutOfRange]);
|
||||
(await TagRowsAsync(rig, "only-pre")).ShouldBe(0);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [pre], default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [pre], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -162,15 +165,15 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1), V(T0.AddSeconds(1), 2)], default);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1), V(T0.AddSeconds(1), 2)], UpdateTests.Who, default);
|
||||
var codes = await rig.Updater.DeleteAtTimeAsync("t",
|
||||
[T0, T0.AddSeconds(5), T0, DateTime.MinValue], default);
|
||||
[T0, T0.AddSeconds(5), T0, DateTime.MinValue], UpdateTests.Who, default);
|
||||
codes.ShouldBe([Good, HistorianStatusCodes.BadNoEntryExists, HistorianStatusCodes.BadNoEntryExists,
|
||||
HistorianStatusCodes.BadInvalidTimestamp]);
|
||||
(await RowsAsync(rig, "t", liveOnly: true)).Select(r => r.Num).ShouldBe([2d]);
|
||||
// a tombstone is not a live row: deleting it again finds nothing
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0], default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("unknown", [T0], default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("unknown", [T0], UpdateTests.Who, default)).ShouldBe([HistorianStatusCodes.BadNoEntryExists]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -178,9 +181,9 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], default);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1)], UpdateTests.Who, default);
|
||||
// +7 ticks is inside the same µs; the row lives at T0.
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0.AddTicks(7)], default)).ShouldBe([Good]);
|
||||
(await rig.Updater.DeleteAtTimeAsync("t", [T0.AddTicks(7)], UpdateTests.Who, default)).ShouldBe([Good]);
|
||||
(await RowsAsync(rig, "t", liveOnly: true)).ShouldBeEmpty();
|
||||
}
|
||||
|
||||
@@ -189,18 +192,18 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
(await rig.Updater.DeleteRawAsync("unknown", T0, T0.AddHours(1), default)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await rig.Updater.DeleteRawAsync("unknown", T0, T0.AddHours(1), UpdateTests.Who, default)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
|
||||
// 2,500 rows => 3 pages of at most 1,000; plus one row exactly at the (exclusive) end.
|
||||
var values = Enumerable.Range(0, 2500).Select(i => V(T0.AddSeconds(i), i)).ToList();
|
||||
values.Add(V(T0.AddSeconds(2500), -1));
|
||||
(await rig.Updater.UpdateDataAsync("big", HistoryUpdateMode.Insert, values, default))
|
||||
(await rig.Updater.UpdateDataAsync("big", HistoryUpdateMode.Insert, values, UpdateTests.Who, default))
|
||||
.ShouldAllBe(c => c == HistorianStatusCodes.GoodEntryInserted);
|
||||
(await rig.Updater.DeleteRawAsync("big", T0, T0.AddSeconds(2500), default)).ShouldBe(Good);
|
||||
(await rig.Updater.DeleteRawAsync("big", T0, T0.AddSeconds(2500), UpdateTests.Who, default)).ShouldBe(Good);
|
||||
var live = await RowsAsync(rig, "big", liveOnly: true);
|
||||
live.Count.ShouldBe(1);
|
||||
live[0].Num.ShouldBe(-1d);
|
||||
(await rig.Updater.DeleteRawAsync("big", T0, T0.AddSeconds(2500), default)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
(await rig.Updater.DeleteRawAsync("big", T0, T0.AddSeconds(2500), UpdateTests.Who, default)).ShouldBe(HistorianStatusCodes.BadNoData);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -208,12 +211,12 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var rig = MakeRig(fx);
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1), V(T0.AddSeconds(1), 2)], default);
|
||||
(await rig.Updater.DeleteRawAsync("t", new DateTime(1900, 1, 1, 0, 0, 0, DateTimeKind.Utc), T0.AddSeconds(1), default))
|
||||
await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T0, 1), V(T0.AddSeconds(1), 2)], UpdateTests.Who, default);
|
||||
(await rig.Updater.DeleteRawAsync("t", new DateTime(1900, 1, 1, 0, 0, 0, DateTimeKind.Utc), T0.AddSeconds(1), UpdateTests.Who, default))
|
||||
.ShouldBe(Good);
|
||||
(await RowsAsync(rig, "t", liveOnly: true)).Select(r => r.Num).ShouldBe([2d]);
|
||||
// DateTime.MinValue as the start is nudged inside, not bound as -infinity
|
||||
(await rig.Updater.DeleteRawAsync("t", DateTime.MinValue, DateTime.MaxValue, default)).ShouldBe(Good);
|
||||
(await rig.Updater.DeleteRawAsync("t", DateTime.MinValue, DateTime.MaxValue, UpdateTests.Who, default)).ShouldBe(Good);
|
||||
(await RowsAsync(rig, "t", liveOnly: true)).ShouldBeEmpty();
|
||||
}
|
||||
|
||||
@@ -226,9 +229,9 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
for (var i = 0; i < 40; i++)
|
||||
{
|
||||
var t = T0.AddSeconds(i);
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(t, i)], default))
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(t, i)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(t, i + 1000)], default))
|
||||
(await rig.Updater.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(t, i + 1000)], UpdateTests.Who, default))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryReplaced], $"iteration {i}: the insert was not visible");
|
||||
}
|
||||
}
|
||||
@@ -242,8 +245,8 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
{
|
||||
var t = T0.AddSeconds(i);
|
||||
var codes = await Task.WhenAll(
|
||||
Task.Run(() => rig.Updater.UpdateDataAsync("race", HistoryUpdateMode.Insert, [V(t, 1)], default)),
|
||||
Task.Run(() => rig.Updater.UpdateDataAsync("race", HistoryUpdateMode.Insert, [V(t, 2)], default)));
|
||||
Task.Run(() => rig.Updater.UpdateDataAsync("race", HistoryUpdateMode.Insert, [V(t, 1)], UpdateTests.Who, default)),
|
||||
Task.Run(() => rig.Updater.UpdateDataAsync("race", HistoryUpdateMode.Insert, [V(t, 2)], UpdateTests.Who, default)));
|
||||
codes.Select(c => c[0]).OrderBy(c => c).ToArray().ShouldBe(
|
||||
new[] { HistorianStatusCodes.GoodEntryInserted, HistorianStatusCodes.BadEntryExists }, $"iteration {i}");
|
||||
}
|
||||
@@ -256,10 +259,10 @@ public sealed class UpdateTests(QuestDbFixture fx)
|
||||
var rig = MakeRig(fx);
|
||||
// 1 Hz seed across 2 h = 7,200 live rows; the request touches 3 instants 1 h apart.
|
||||
var seed = Enumerable.Range(0, 7200).Select(i => V(T0.AddSeconds(i), i)).ToList();
|
||||
await rig.Updater.UpdateDataAsync("sparse", HistoryUpdateMode.Insert, seed, default);
|
||||
await rig.Updater.UpdateDataAsync("sparse", HistoryUpdateMode.Insert, seed, UpdateTests.Who, default);
|
||||
var before = rig.Updater.LiveRowsRead;
|
||||
var codes = await rig.Updater.UpdateDataAsync("sparse", HistoryUpdateMode.Replace,
|
||||
[V(T0, 1), V(T0.AddHours(1), 2), V(T0.AddHours(2).AddSeconds(-1), 3), V(T0.AddHours(1).AddMilliseconds(500), 4)], default);
|
||||
[V(T0, 1), V(T0.AddHours(1), 2), V(T0.AddHours(2).AddSeconds(-1), 3), V(T0.AddHours(1).AddMilliseconds(500), 4)], UpdateTests.Who, default);
|
||||
codes.ShouldBe([HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.GoodEntryReplaced,
|
||||
HistorianStatusCodes.GoodEntryReplaced, HistorianStatusCodes.BadNoEntryExists]);
|
||||
// 3 clusters of a single instant each (T0+1h and +1h.5s are one cluster of 2 instants, 1 live row)
|
||||
|
||||
+13
-1
@@ -39,10 +39,22 @@ public sealed class QuestDbTablesTests
|
||||
[Fact]
|
||||
public void The_prefix_is_applied_to_every_table_name()
|
||||
{
|
||||
QuestDbTables.All("t1_").ShouldBe(["t1_sample", "t1_event", "t1_tag"]);
|
||||
QuestDbTables.All("t1_").ShouldBe(["t1_sample", "t1_event", "t1_tag", "t1_sample_modified"]);
|
||||
QuestDbTables.Expiring("t1_").ShouldBe(["t1_sample", "t1_event", "t1_sample_modified"]);
|
||||
QuestDbTables.CreateAll("t1_", 3).ShouldAllBe(s => s.Contains("t1_"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Sample_modified_ddl_is_pinned_append_only_and_carries_the_ttl()
|
||||
{
|
||||
QuestDbTables.CreateSampleModified("hist_", null).ShouldBe(
|
||||
"CREATE TABLE IF NOT EXISTS hist_sample_modified (ts TIMESTAMP, tag SYMBOL, modified_id LONG, " +
|
||||
"modification_ts TIMESTAMP, update_type BYTE, principal VARCHAR, api_key_id VARCHAR, kind BYTE, num_value DOUBLE, " +
|
||||
"int_value LONG, str_value VARCHAR, status LONG, server_ts TIMESTAMP) TIMESTAMP(ts) PARTITION BY DAY WAL");
|
||||
QuestDbTables.CreateSampleModified("hist_", 30).ShouldEndWith("PARTITION BY DAY TTL 30 DAYS WAL");
|
||||
QuestDbTables.CreateSampleModified("hist_", null).ShouldNotContain("DEDUP");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void An_invalid_prefix_is_refused() =>
|
||||
Should.Throw<ArgumentException>(() => QuestDbTables.CreateSample("a b", null));
|
||||
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
using Microsoft.Data.SqlClient;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Conformance;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.Historian.Sql.Common;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// #668, SQL Server only: the modification-trail row is written by the SAME statement that changes
|
||||
/// <c>hist.Sample</c> (<c>OUTPUT … INTO</c>), so a change and its trail row commit or roll back together. The failure
|
||||
/// is forced INSIDE the audit insert (a unique index on the trail that the second modification of one timestamp
|
||||
/// violates), i.e. after the sample row was already changed within the statement — the strongest place to prove it.
|
||||
/// Each test has a positive control: the same operation succeeds, with its trail row, once the index is dropped.
|
||||
/// </summary>
|
||||
[Collection(SqlHistorianCollection.Name)]
|
||||
public sealed class ModifiedHistoryAtomicityTests(SqlHistorianFixture fx)
|
||||
{
|
||||
private static readonly DateTime T1 = new(2026, 1, 1, 0, 0, 1, DateTimeKind.Utc);
|
||||
private static readonly DateTime T2 = T1.AddSeconds(1);
|
||||
private static readonly HistoryUpdateContext Who = ConformanceData.Editor;
|
||||
|
||||
private static HistorizationValue V(DateTime t, double d) => new(t, HistorianValue.FromDouble(d), 0u);
|
||||
|
||||
private async Task ExecAsync(ReadTestDb db, string sql)
|
||||
{
|
||||
await using var c = await db.Conn.OpenAsync(CancellationToken.None);
|
||||
await using var cmd = db.Conn.CreateCommand(c, sql);
|
||||
await cmd.ExecuteNonQueryAsync();
|
||||
}
|
||||
|
||||
private async Task<long> CountTrailAsync(ReadTestDb db)
|
||||
{
|
||||
await using var c = await db.Conn.OpenAsync(CancellationToken.None);
|
||||
await using var cmd = db.Conn.CreateCommand(c, $"SELECT COUNT(*) FROM {db.Conn.Q("SampleModified")}");
|
||||
return Convert.ToInt64(await cmd.ExecuteScalarAsync());
|
||||
}
|
||||
|
||||
private Task BreakTheTrailAsync(ReadTestDb db) => ExecAsync(db,
|
||||
$"CREATE UNIQUE INDEX UX_Break ON {db.Conn.Q("SampleModified")} (TagId, SourceTimeUtc)");
|
||||
|
||||
private Task RepairTheTrailAsync(ReadTestDb db) => ExecAsync(db,
|
||||
$"DROP INDEX UX_Break ON {db.Conn.Q("SampleModified")}");
|
||||
|
||||
private static async Task<double> ValueAtAsync(ReadTestDb db, string tag, DateTime t)
|
||||
{
|
||||
using var src = db.Source();
|
||||
var r = await src.ReadRawAsync(tag, t, t, 0, CancellationToken.None);
|
||||
return (double)r.Samples.ShouldHaveSingleItem().Value!;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_replace_whose_trail_insert_fails_changes_nothing()
|
||||
{
|
||||
var db = await ReadTestDb.CreateAsync(fx);
|
||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], Who, CancellationToken.None))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
(await CountTrailAsync(db)).ShouldBe(1);
|
||||
|
||||
await BreakTheTrailAsync(db);
|
||||
await Should.ThrowAsync<SqlException>(
|
||||
() => u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(T1, 99)], Who, CancellationToken.None));
|
||||
|
||||
(await ValueAtAsync(db, "t", T1)).ShouldBe(1d, "the sample change rolled back with its trail row");
|
||||
(await CountTrailAsync(db)).ShouldBe(1, "a failed update leaves no trail row");
|
||||
|
||||
// Positive control: the identical replace succeeds, with its trail row, once the trail accepts it.
|
||||
await RepairTheTrailAsync(db);
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Replace, [V(T1, 99)], Who, CancellationToken.None))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryReplaced]);
|
||||
(await ValueAtAsync(db, "t", T1)).ShouldBe(99d);
|
||||
(await CountTrailAsync(db)).ShouldBe(2);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Deletes_whose_trail_insert_fails_delete_nothing()
|
||||
{
|
||||
var db = await ReadTestDb.CreateAsync(fx);
|
||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||
await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1), V(T2, 2)], Who, CancellationToken.None);
|
||||
|
||||
await BreakTheTrailAsync(db);
|
||||
await Should.ThrowAsync<SqlException>(() => u.DeleteAtTimeAsync("t", [T1], Who, CancellationToken.None));
|
||||
await Should.ThrowAsync<SqlException>(() => u.DeleteRawAsync("t", T1, T2.AddSeconds(1), Who, CancellationToken.None));
|
||||
|
||||
(await ValueAtAsync(db, "t", T1)).ShouldBe(1d);
|
||||
(await ValueAtAsync(db, "t", T2)).ShouldBe(2d);
|
||||
(await CountTrailAsync(db)).ShouldBe(2, "only the two inserts");
|
||||
|
||||
await RepairTheTrailAsync(db);
|
||||
(await u.DeleteAtTimeAsync("t", [T1], Who, CancellationToken.None)).ShouldBe([HistorianStatusCodes.Good]);
|
||||
(await u.DeleteRawAsync("t", T1, T2.AddSeconds(1), Who, CancellationToken.None)).ShouldBe(HistorianStatusCodes.Good);
|
||||
(await CountTrailAsync(db)).ShouldBe(4);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_over_long_principal_is_truncated_never_refused()
|
||||
{
|
||||
var db = await ReadTestDb.CreateAsync(fx);
|
||||
var u = new SqlServerHistorianDataUpdater(db.Conn, new SqlServerTagCatalog(db.Conn, db.Boot));
|
||||
var who = new HistoryUpdateContext(new string('p', 500), new string('k', 500));
|
||||
|
||||
(await u.UpdateDataAsync("t", HistoryUpdateMode.Insert, [V(T1, 1)], who, CancellationToken.None))
|
||||
.ShouldBe([HistorianStatusCodes.GoodEntryInserted]);
|
||||
|
||||
using var src = db.Source();
|
||||
var trail = await src.ReadModifiedAsync("t", T1.AddHours(-1), T1.AddHours(1), 0, null, CancellationToken.None);
|
||||
trail.Values.ShouldHaveSingleItem().UserName.ShouldBe(new string('p', HistoryUpdateContext.MaxIdentityLength));
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -30,7 +30,7 @@ public sealed class RawReadTests(SqlHistorianFixture fx)
|
||||
var h = src.GetHealthSnapshot();
|
||||
h.TotalFailures.ShouldBe(2);
|
||||
h.ConsecutiveFailures.ShouldBe(2);
|
||||
h.LastError!.ShouldContain("missing version 1");
|
||||
h.LastError!.ShouldContain($"missing version {SqlServerSchemaBootstrapper.RequiredVersion}");
|
||||
h.ProcessConnectionOpen.ShouldBeFalse();
|
||||
|
||||
// Another node creates the schema; the same data source then succeeds (bootstrap failure is not latched).
|
||||
|
||||
+20
-2
@@ -56,6 +56,9 @@ public sealed class RetentionTests(SqlHistorianFixture fx)
|
||||
sql.Append($"SET @i = 0; WHILE @i < 2 BEGIN INSERT [{s}].Sample (TagId, SourceTimeUtc, ValueKind, NumValue, StatusCode, ServerTimeUtc) SELECT TagId, DATEADD(minute, @i, '{fresh}'), 1, 1.0, 0, SYSUTCDATETIME() FROM [{s}].Tag; SET @i += 1; END;");
|
||||
sql.Append($"SET @i = 0; WHILE @i < 4 BEGIN INSERT [{s}].Event (EventId, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, EventTimeUtc, ReceivedTimeUtc) VALUES (CONCAT('old', @i), 's', 'a', 'n', 't', 1, 'Activated', DATEADD(minute, @i, '{old}'), SYSUTCDATETIME()); SET @i += 1; END;");
|
||||
sql.Append($"INSERT [{s}].Event (EventId, SourceName, AlarmId, AlarmName, AlarmTypeName, Severity, EventKind, EventTimeUtc, ReceivedTimeUtc) VALUES ('new0', 's', 'a', 'n', 't', 1, 'Activated', '{fresh}', SYSUTCDATETIME());");
|
||||
// #668 modification trail: 2 old + 1 fresh row per tag (by SOURCE time — the modification time is "now" on all).
|
||||
sql.Append($"SET @i = 0; WHILE @i < 2 BEGIN INSERT [{s}].SampleModified (TagId, SourceTimeUtc, ModificationTimeUtc, UpdateType, Principal, ValueKind, NumValue, StatusCode, ServerTimeUtc) SELECT TagId, DATEADD(minute, @i, '{old}'), SYSUTCDATETIME(), 2, N'p', 1, 1.0, 0, SYSUTCDATETIME() FROM [{s}].Tag; SET @i += 1; END;");
|
||||
sql.Append($"INSERT [{s}].SampleModified (TagId, SourceTimeUtc, ModificationTimeUtc, UpdateType, Principal) SELECT TagId, '{fresh}', SYSUTCDATETIME(), 1, N'p' FROM [{s}].Tag;");
|
||||
await ExecAsync(sql.ToString());
|
||||
}
|
||||
|
||||
@@ -67,11 +70,22 @@ public sealed class RetentionTests(SqlHistorianFixture fx)
|
||||
fx.SkipIfUnavailable();
|
||||
var (c, b, _, s) = Make();
|
||||
await SeedAsync(b, s);
|
||||
var identBefore = await CountAsync($"SELECT IDENT_CURRENT('[{s}].SampleModified')");
|
||||
|
||||
var purger = new SqlServerHistorianRetention(c, b, TimeSpan.FromHours(1), 30, batch, time: new FixedTime(Now));
|
||||
var results = await purger.PurgeAsync(CancellationToken.None);
|
||||
|
||||
results.ShouldBe([new RetentionPurgeResult("Sample", 15), new RetentionPurgeResult("Event", 4)]);
|
||||
// #668: a purge never WRITES to the modification trail. The identity seed proves no row was ever inserted —
|
||||
// a row-count check could not, since the trail purge would delete an old-source-time audit row again.
|
||||
(await CountAsync($"SELECT IDENT_CURRENT('[{s}].SampleModified')")).ShouldBe(identBefore);
|
||||
|
||||
results.ShouldBe([
|
||||
new RetentionPurgeResult("Sample", 15),
|
||||
new RetentionPurgeResult("SampleModified", 6),
|
||||
new RetentionPurgeResult("Event", 4),
|
||||
]);
|
||||
(await CountAsync($"SELECT COUNT(*) FROM [{s}].SampleModified")).ShouldBe(3);
|
||||
(await CountAsync($"SELECT COUNT(*) FROM [{s}].SampleModified WHERE SourceTimeUtc < '{Now.AddDays(-30):yyyy-MM-ddTHH:mm:ss}'")).ShouldBe(0);
|
||||
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Sample")).ShouldBe(6);
|
||||
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Sample WHERE SourceTimeUtc < '{Now.AddDays(-30):yyyy-MM-ddTHH:mm:ss}'")).ShouldBe(0);
|
||||
(await CountAsync($"SELECT COUNT(*) FROM [{s}].Event")).ShouldBe(1);
|
||||
@@ -87,7 +101,11 @@ public sealed class RetentionTests(SqlHistorianFixture fx)
|
||||
var purger = new SqlServerHistorianRetention(c, b, TimeSpan.FromHours(1), 30, 4000, time: new FixedTime(Now));
|
||||
await purger.PurgeAsync(CancellationToken.None);
|
||||
var again = await purger.PurgeAsync(CancellationToken.None);
|
||||
again.ShouldBe([new RetentionPurgeResult("Sample", 0), new RetentionPurgeResult("Event", 0)]);
|
||||
again.ShouldBe([
|
||||
new RetentionPurgeResult("Sample", 0),
|
||||
new RetentionPurgeResult("SampleModified", 0),
|
||||
new RetentionPurgeResult("Event", 0),
|
||||
]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+44
-4
@@ -32,7 +32,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
||||
var s = SqlHistorianFixture.UniqueSchema();
|
||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None);
|
||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None); // fresh instance => re-runs the guarded path
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(1);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM sys.indexes WHERE name='CX_Event_Time' AND object_id=OBJECT_ID('[{s}].Event')")).ShouldBe(1);
|
||||
}
|
||||
|
||||
@@ -51,7 +51,47 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
||||
cmd.CommandText = b;
|
||||
await cmd.ExecuteNonQueryAsync();
|
||||
}
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(1);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #668 upgrade path: a schema left at version 1 (only <c>001_initial.sql</c> applied, as every pre-#668 install is)
|
||||
/// is upgraded in place by an auto-creating bootstrap — <c>SampleModified</c> appears, both version rows exist, and
|
||||
/// existing samples are untouched. With <c>AutoCreateSchema=false</c> the same v1 schema is refused.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task A_version_1_schema_is_upgraded_to_version_2_in_place()
|
||||
{
|
||||
fx.SkipIfUnavailable();
|
||||
var s = SqlHistorianFixture.UniqueSchema();
|
||||
var v1 = new StreamReader(typeof(SqlServerSchemaBootstrapper).Assembly.GetManifestResourceStream("Schema.001_initial.sql")!).ReadToEnd();
|
||||
await using (var c = new SqlConnection(fx.ConnectionString))
|
||||
{
|
||||
await c.OpenAsync();
|
||||
foreach (var b in SqlServerSchemaBootstrapper.SplitBatches(v1.Replace("$(Schema)", s)))
|
||||
{
|
||||
await using var cmd = c.CreateCommand();
|
||||
cmd.CommandText = b;
|
||||
await cmd.ExecuteNonQueryAsync();
|
||||
}
|
||||
await using var seed = c.CreateCommand();
|
||||
seed.CommandText = $"INSERT [{s}].Tag (Name, DataType, CreatedUtc, UpdatedUtc) VALUES (N'old', 1, SYSUTCDATETIME(), SYSUTCDATETIME()); " +
|
||||
$"INSERT [{s}].Sample (TagId, SourceTimeUtc, ValueKind, NumValue, StatusCode, ServerTimeUtc) SELECT TagId, '2026-01-01', 1, 4.0, 0, SYSUTCDATETIME() FROM [{s}].Tag";
|
||||
await seed.ExecuteNonQueryAsync();
|
||||
}
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM sys.tables WHERE name = 'SampleModified' AND schema_id = SCHEMA_ID('{s}')")).ShouldBe(0);
|
||||
|
||||
var refused = await Should.ThrowAsync<HistorianSchemaMissingException>(
|
||||
() => Boot(Opts(s, auto: false)).EnsureAsync(CancellationToken.None));
|
||||
refused.Message.ShouldContain("missing version 2 (found 1)");
|
||||
|
||||
await Boot(Opts(s)).EnsureAsync(CancellationToken.None);
|
||||
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
||||
(await ScalarAsync($"SELECT MAX(Version) FROM [{s}].SchemaVersion")).ShouldBe(2);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SampleModified")).ShouldBe(0);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].Sample")).ShouldBe(1);
|
||||
await Boot(Opts(s, auto: false)).EnsureAsync(CancellationToken.None); // now satisfies the verify-only path
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -61,7 +101,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
||||
var s = SqlHistorianFixture.UniqueSchema();
|
||||
var tasks = Enumerable.Range(0, 4).Select(_ => Task.Run(() => Boot(Opts(s)).EnsureAsync(CancellationToken.None))).ToArray();
|
||||
await Task.WhenAll(tasks);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(1);
|
||||
(await ScalarAsync($"SELECT COUNT(*) FROM [{s}].SchemaVersion")).ShouldBe(SqlServerSchemaBootstrapper.RequiredVersion);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -70,7 +110,7 @@ public sealed class SchemaBootstrapIntegrationTests(SqlHistorianFixture fx)
|
||||
fx.SkipIfUnavailable();
|
||||
var ex = await Should.ThrowAsync<HistorianSchemaMissingException>(
|
||||
() => Boot(Opts(SqlHistorianFixture.UniqueSchema(), auto: false)).EnsureAsync(CancellationToken.None));
|
||||
ex.Message.ShouldContain("missing version 1");
|
||||
ex.Message.ShouldContain("missing version 2");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+7
@@ -53,3 +53,10 @@ public sealed class SqlServerTagNameConformance(SqlHistorianFixture fx) : TagNam
|
||||
/// <inheritdoc/>
|
||||
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
|
||||
}
|
||||
|
||||
[Collection(SqlHistorianCollection.Name)]
|
||||
public sealed class SqlServerModifiedValueConformance(SqlHistorianFixture fx) : ModifiedValueConformance
|
||||
{
|
||||
/// <inheritdoc/>
|
||||
protected override Task<IConformanceBackend> CreateBackendAsync() => SqlServerConformanceBackend.CreateAsync(fx);
|
||||
}
|
||||
|
||||
@@ -29,4 +29,30 @@ public sealed class SchemaScriptTests
|
||||
dir.ShouldNotBeNull();
|
||||
File.ReadAllText(Path.Combine(dir!.FullName, "docs/historian/sqlserver/001_initial.sql")).ShouldBe(text);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #668: schema v2 (the Part 11 modification trail) is embedded, matches its docs copy, records version 2 (which
|
||||
/// must equal <see cref="SqlServerSchemaBootstrapper.RequiredVersion"/>), and carries none of the constructs an
|
||||
/// <c>OUTPUT … INTO</c> target forbids (FOREIGN KEY / CHECK / trigger) — any of them would fail every HistoryUpdate.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Schema_v2_sample_modified_is_embedded_matches_the_docs_copy_and_is_a_legal_output_into_target()
|
||||
{
|
||||
var asm = typeof(SqlServerSchemaBootstrapper).Assembly;
|
||||
using var s = asm.GetManifestResourceStream("Schema.002_sample_modified.sql");
|
||||
s.ShouldNotBeNull();
|
||||
var text = new StreamReader(s!).ReadToEnd();
|
||||
text.ShouldContain("[SampleModified]");
|
||||
text.ShouldContain("PRIMARY KEY CLUSTERED (TagId, SourceTimeUtc, ModifiedId)");
|
||||
text.ShouldContain("VALUES (2, SYSUTCDATETIME())");
|
||||
SqlServerSchemaBootstrapper.RequiredVersion.ShouldBe(2);
|
||||
var ddl = string.Join('\n', text.Split('\n').Where(l => !l.TrimStart().StartsWith("--", StringComparison.Ordinal)));
|
||||
ddl.ShouldNotContain("FOREIGN KEY", Case.Insensitive);
|
||||
ddl.ShouldNotContain("CHECK", Case.Insensitive);
|
||||
ddl.ShouldNotContain("TRIGGER", Case.Insensitive);
|
||||
|
||||
var dir = new DirectoryInfo(AppContext.BaseDirectory);
|
||||
while (dir is not null && !File.Exists(Path.Combine(dir.FullName, "ZB.MOM.WW.OtOpcUa.slnx"))) dir = dir.Parent;
|
||||
File.ReadAllText(Path.Combine(dir!.FullName, "docs/historian/sqlserver/002_sample_modified.sql")).ShouldBe(text);
|
||||
}
|
||||
}
|
||||
|
||||
+77
-4
@@ -72,6 +72,50 @@ public sealed class HistoryUpdateOverTheWireTests
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Part 11 modified-value history over the wire (#668): two inserts and a delete by the key principal, then a
|
||||
/// HistoryReadRaw with <c>IsReadModified = true</c> decodes as a <see cref="HistoryModifiedData"/> whose
|
||||
/// <c>ModificationInfos</c> carry the update type and the principal, and whose values carry the PRIOR value.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task A_modified_read_returns_the_trail_with_principal_and_update_type_over_the_wire()
|
||||
{
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var store = new SharedStore();
|
||||
var rig = await StartAsync("HuWireModified", encrypted: false, ct);
|
||||
try
|
||||
{
|
||||
rig.Nm.HistorianDataUpdater = store;
|
||||
rig.Nm.HistorianDataSource = store;
|
||||
rig.Enforce(true, [Key()], ClusterGrant(KeyPrincipal, "acl-editor", Editor));
|
||||
var id = rig.EnsureRawVariable("Speed", "Double", writable: false, Tagname);
|
||||
|
||||
using var session = await rig.ConnectAsync(anonymous: false, ct);
|
||||
var node = RawNode(session, id);
|
||||
await UpdateAsync(session, node, PerformUpdateType.Insert, ct, (T1, 1.5), (T2, 2.5));
|
||||
await DeleteAtTimeAsync(session, node, ct, T1);
|
||||
|
||||
var details = new ReadRawModifiedDetails
|
||||
{
|
||||
StartTime = T1.AddHours(-1), EndTime = T1.AddHours(1), NumValuesPerNode = 100, IsReadModified = true,
|
||||
};
|
||||
var response = await session.HistoryReadAsync(
|
||||
null, new ExtensionObject(details), TimestampsToReturn.Both, false,
|
||||
new HistoryReadValueIdCollection { new HistoryReadValueId { NodeId = node } }, ct);
|
||||
var result = response.Results.ShouldHaveSingleItem();
|
||||
result.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||
var data = ExtensionObject.ToEncodeable(result.HistoryData).ShouldBeOfType<HistoryModifiedData>();
|
||||
|
||||
data.DataValues.Select(v => v.SourceTimestamp).ShouldBe([T1, T1, T2]);
|
||||
data.ModificationInfos.Select(m => m.UpdateType)
|
||||
.ShouldBe([HistoryUpdateType.Insert, HistoryUpdateType.Delete, HistoryUpdateType.Insert]);
|
||||
data.ModificationInfos.ShouldAllBe(m => m.UserName == KeyPrincipal);
|
||||
data.DataValues[0].StatusCode.Code.ShouldBe(StatusCodes.BadNoData); // an insert had no prior value
|
||||
((double)data.DataValues[1].Value).ShouldBe(1.5); // the deleted sample's prior value
|
||||
}
|
||||
finally { await rig.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ALLOW beside DENY: the same server, same node — the editor session is admitted, the session
|
||||
/// without the HistoryUpdate grant gets BadUserAccessDenied and the updater is never called.
|
||||
@@ -263,14 +307,21 @@ public sealed class HistoryUpdateOverTheWireTests
|
||||
{
|
||||
private readonly object _gate = new();
|
||||
private readonly SortedDictionary<DateTime, double> _samples = [];
|
||||
private readonly List<HistoryModifiedValue> _trail = [];
|
||||
private int _updateCalls;
|
||||
|
||||
private void Record(DateTime t, double? prior, HistoryModificationType type, HistoryUpdateContext who) =>
|
||||
_trail.Add(new HistoryModifiedValue(
|
||||
new DataValueSnapshot(prior, prior is null ? StatusCodes.BadNoData : StatusCodes.Good, t, t),
|
||||
DateTime.UtcNow, type, who.Principal, _trail.Count + 1));
|
||||
|
||||
public int UpdateCalls => Volatile.Read(ref _updateCalls);
|
||||
|
||||
public int Count { get { lock (_gate) return _samples.Count; } }
|
||||
|
||||
public async Task<IReadOnlyList<uint>> UpdateDataAsync(
|
||||
string tag, HistoryUpdateMode mode, IReadOnlyList<HistorizationValue> values, CancellationToken ct)
|
||||
string tag, HistoryUpdateMode mode, IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
Interlocked.Increment(ref _updateCalls);
|
||||
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||
@@ -285,21 +336,43 @@ public sealed class HistoryUpdateOverTheWireTests
|
||||
if (_samples.ContainsKey(t)) { codes.Add(StatusCodes.BadEntryExists); continue; }
|
||||
v.Value.TryGetDouble(out var d).ShouldBeTrue();
|
||||
_samples[t] = d;
|
||||
Record(t, null, HistoryModificationType.Insert, context);
|
||||
codes.Add(StatusCodes.GoodEntryInserted);
|
||||
}
|
||||
}
|
||||
return codes;
|
||||
}
|
||||
|
||||
public Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct) =>
|
||||
public Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct) =>
|
||||
throw new NotSupportedException();
|
||||
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(
|
||||
string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct)
|
||||
string tag, IReadOnlyList<DateTime> timesUtc, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||
lock (_gate)
|
||||
return timesUtc.Select(t => _samples.Remove(t) ? StatusCodes.Good : StatusCodes.BadNoEntryExists).ToList();
|
||||
return timesUtc.Select(t =>
|
||||
{
|
||||
if (!_samples.Remove(t, out var prior)) return StatusCodes.BadNoEntryExists;
|
||||
Record(t, prior, HistoryModificationType.Delete, context);
|
||||
return StatusCodes.Good;
|
||||
}).ToList();
|
||||
}
|
||||
|
||||
public async Task<HistoryModifiedReadResult> ReadModifiedAsync(
|
||||
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValues, HistoryModifiedCursor? resumeAfter,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
await Task.Delay(1, cancellationToken).ConfigureAwait(false);
|
||||
fullReference.ShouldBe(Tagname);
|
||||
lock (_gate)
|
||||
{
|
||||
var rows = _trail
|
||||
.Where(m => m.Value.SourceTimestampUtc >= startUtc && m.Value.SourceTimestampUtc < endUtc)
|
||||
.OrderBy(m => m.Value.SourceTimestampUtc).ThenBy(m => m.ModifiedId).ToList();
|
||||
return new HistoryModifiedReadResult(rows, false);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<CoreHistoryReadResult> ReadRawAsync(
|
||||
|
||||
@@ -463,6 +463,56 @@ public sealed class HistoryUpdateTests : IDisposable
|
||||
finally { await host.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #668: the session's API-key principal and key id reach EVERY updater call (they become the Part 11
|
||||
/// ModificationInfo.UserName of the trail rows the backend records).
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task The_key_principal_and_key_id_are_threaded_to_every_updater_call()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
try
|
||||
{
|
||||
var nm = server.NodeManager!;
|
||||
var updater = new FakeUpdater();
|
||||
nm.HistorianDataUpdater = updater;
|
||||
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||
var identity = KeyIdentity(OpcUaDataPlaneRoles.HistoryUpdate);
|
||||
|
||||
Update(nm, identity, nodeId, PerformUpdateType.Update, Value(T1, 1d));
|
||||
Call(nm, identity, typeof(DeleteRawModifiedDetails),
|
||||
new DeleteRawModifiedDetails { NodeId = nodeId, StartTime = T1, EndTime = T2, IsDeleteModified = false });
|
||||
Call(nm, identity, typeof(DeleteAtTimeDetails),
|
||||
new DeleteAtTimeDetails { NodeId = nodeId, ReqTimes = new DateTimeCollection { T1 } });
|
||||
|
||||
updater.Callers.Count.ShouldBe(3);
|
||||
updater.Callers.ShouldAllBe(c => c == new HistoryUpdateContext(Principal, KeyId));
|
||||
}
|
||||
finally { await host.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>#668: an admitted anonymous session is recorded as the reserved <c>anonymous</c> principal, no key id.</summary>
|
||||
[Fact]
|
||||
public async Task An_anonymous_session_is_recorded_as_the_anonymous_principal()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
try
|
||||
{
|
||||
var nm = server.NodeManager!;
|
||||
var updater = new FakeUpdater();
|
||||
nm.HistorianDataUpdater = updater;
|
||||
WireAcl(nm, enabled: false, NodePermissions.HistoryUpdate);
|
||||
var nodeId = Historized(nm, AddressSpaceRealm.Raw, RawPath);
|
||||
|
||||
var (_, errors) = Insert(nm, AnonymousIdentity(OpcUaDataPlaneRoles.HistoryUpdate), nodeId, Value(T1, 1d));
|
||||
|
||||
errors[0].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||
updater.Callers.ShouldHaveSingleItem().ShouldBe(new HistoryUpdateContext(OpcUaPrincipals.Anonymous, null));
|
||||
}
|
||||
finally { await host.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>DeleteAtTime returns the updater's per-time codes in order.</summary>
|
||||
[Fact]
|
||||
public async Task Delete_at_time_passes_per_time_codes_through()
|
||||
@@ -593,6 +643,7 @@ public sealed class HistoryUpdateTests : IDisposable
|
||||
private sealed class FakeUpdater : IHistorianDataUpdater
|
||||
{
|
||||
public List<(string Tag, HistoryUpdateMode Mode, IReadOnlyList<HistorizationValue> Values)> UpdateCalls { get; } = [];
|
||||
public List<HistoryUpdateContext> Callers { get; } = [];
|
||||
public List<(string Tag, DateTime Start, DateTime End)> DeleteRawCalls { get; } = [];
|
||||
public List<IReadOnlyList<DateTime>> DeleteAtTimeCalls { get; } = [];
|
||||
public uint[]? UpdateCodes { get; init; }
|
||||
@@ -601,26 +652,28 @@ public sealed class HistoryUpdateTests : IDisposable
|
||||
public Exception? Throw { get; init; }
|
||||
|
||||
public async Task<IReadOnlyList<uint>> UpdateDataAsync(
|
||||
string tag, HistoryUpdateMode mode, IReadOnlyList<HistorizationValue> values, CancellationToken ct)
|
||||
string tag, HistoryUpdateMode mode, IReadOnlyList<HistorizationValue> values, HistoryUpdateContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||
if (Throw is not null) throw Throw;
|
||||
lock (UpdateCalls) UpdateCalls.Add((tag, mode, values.ToList()));
|
||||
lock (UpdateCalls) { UpdateCalls.Add((tag, mode, values.ToList())); Callers.Add(context); }
|
||||
return UpdateCodes ?? Enumerable.Repeat(StatusCodes.GoodEntryInserted, values.Count).ToArray();
|
||||
}
|
||||
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, CancellationToken ct)
|
||||
public async Task<uint> DeleteRawAsync(string tag, DateTime startUtc, DateTime endUtc, HistoryUpdateContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||
lock (DeleteRawCalls) DeleteRawCalls.Add((tag, startUtc, endUtc));
|
||||
lock (DeleteRawCalls) { DeleteRawCalls.Add((tag, startUtc, endUtc)); lock (UpdateCalls) Callers.Add(context); }
|
||||
return DeleteRawCode;
|
||||
}
|
||||
|
||||
public async Task<IReadOnlyList<uint>> DeleteAtTimeAsync(
|
||||
string tag, IReadOnlyList<DateTime> timesUtc, CancellationToken ct)
|
||||
string tag, IReadOnlyList<DateTime> timesUtc, HistoryUpdateContext context, CancellationToken ct)
|
||||
{
|
||||
await Task.Delay(1, ct).ConfigureAwait(false);
|
||||
lock (DeleteAtTimeCalls) DeleteAtTimeCalls.Add(timesUtc.ToList());
|
||||
lock (DeleteAtTimeCalls) { DeleteAtTimeCalls.Add(timesUtc.ToList()); lock (UpdateCalls) Callers.Add(context); }
|
||||
return DeleteAtTimeCodes ?? Enumerable.Repeat(StatusCodes.Good, timesUtc.Count).ToArray();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -89,6 +89,33 @@ public sealed class NodeManagerAclEnforcementTests : IDisposable
|
||||
finally { await host.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// #668: a modified-value read (<c>IsReadModified = true</c>) is gated by the SAME HistoryRead decision — no new
|
||||
/// operation. Denied without the bit; the granted principal reaches the backend (Null ⇒ unsupported, which is
|
||||
/// "allowed but no trail", a different fact from the denial).
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Modified_history_read_is_gated_by_the_history_read_grant()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
try
|
||||
{
|
||||
var nm = server.NodeManager!;
|
||||
nm.HistorianDataSource = NullHistorianDataSource.Instance;
|
||||
var nodeId = Historized(nm, "Plant/Modbus/dev1/Speed");
|
||||
|
||||
nm.Authorizer = Authorizer(enabled: true, GrantOn(NodePermissions.Read));
|
||||
var (deniedResults, deniedErrors) = ReadRaw(nm, Identity(), modified: true, nodeId);
|
||||
deniedErrors[0].StatusCode.Code.ShouldBe(StatusCodes.BadUserAccessDenied);
|
||||
deniedResults[0].StatusCode.Code.ShouldBe(StatusCodes.BadUserAccessDenied);
|
||||
|
||||
nm.Authorizer = Authorizer(enabled: true, GrantOn(NodePermissions.HistoryRead));
|
||||
var (_, allowedErrors) = ReadRaw(nm, Identity(), modified: true, nodeId);
|
||||
allowedErrors[0].StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||
}
|
||||
finally { await host.DisposeAsync(); }
|
||||
}
|
||||
|
||||
/// <summary>A granted principal reads normally — the gate is not simply denying everything.</summary>
|
||||
[Fact]
|
||||
public async Task History_read_succeeds_for_a_granted_principal()
|
||||
@@ -308,7 +335,11 @@ public sealed class NodeManagerAclEnforcementTests : IDisposable
|
||||
}
|
||||
|
||||
private static (List<SdkHistoryReadResult> Results, List<ServiceResult> Errors) ReadRaw(
|
||||
OtOpcUaNodeManager nm, IUserIdentity? identity, params NodeId[] nodeIds)
|
||||
OtOpcUaNodeManager nm, IUserIdentity? identity, params NodeId[] nodeIds) =>
|
||||
ReadRaw(nm, identity, modified: false, nodeIds);
|
||||
|
||||
private static (List<SdkHistoryReadResult> Results, List<ServiceResult> Errors) ReadRaw(
|
||||
OtOpcUaNodeManager nm, IUserIdentity? identity, bool modified, params NodeId[] nodeIds)
|
||||
{
|
||||
var context = new OperationContext(
|
||||
new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity);
|
||||
@@ -324,7 +355,7 @@ public sealed class NodeManagerAclEnforcementTests : IDisposable
|
||||
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||
EndTime = DateTime.UtcNow,
|
||||
NumValuesPerNode = 10,
|
||||
IsReadModified = false,
|
||||
IsReadModified = modified,
|
||||
},
|
||||
TimestampsToReturn.Both,
|
||||
releaseContinuationPoints: false,
|
||||
|
||||
+308
@@ -0,0 +1,308 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Opc.Ua;
|
||||
using Opc.Ua.Server;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
using HistorianRead = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryReadResult;
|
||||
using SdkHistoryReadResult = Opc.Ua.HistoryReadResult;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #668 — OPC UA Part 11 modified-value history: HistoryReadRaw with <c>IsReadModified = true</c> dispatches
|
||||
/// to <see cref="IHistorianDataSource.ReadModifiedAsync"/>, projects a <see cref="HistoryModifiedData"/> with one
|
||||
/// <see cref="ModificationInfo"/> per value, and pages with continuation points keyed on
|
||||
/// <c>(SourceTimeUtc, ModifiedId)</c>. Same harness as <see cref="NodeManagerHistoryReadEventsPagingTests"/>: a real
|
||||
/// SDK server, a session-less in-process HistoryRead and an <see cref="InMemoryHistoryContinuationStore"/>, over a
|
||||
/// fake that behaves like the SQL backends (ordered by key, half-open window, cap + exact truncation flag).
|
||||
/// </summary>
|
||||
public sealed class NodeManagerHistoryReadModifiedTests : IDisposable
|
||||
{
|
||||
private const string Tag = "WW.Mod";
|
||||
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||
private static readonly DateTime Epoch = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
|
||||
private readonly string _pkiRoot = Path.Combine(Path.GetTempPath(), $"otopcua-modified-{Guid.NewGuid():N}");
|
||||
|
||||
[Fact]
|
||||
public async Task A_modified_read_dispatches_to_the_trail_and_projects_modification_infos()
|
||||
{
|
||||
var modifiedAt = Epoch.AddDays(2);
|
||||
var trail = new List<HistoryModifiedValue>
|
||||
{
|
||||
Mod(1, seconds: 10, prior: null, HistoryModificationType.Insert, "alice", modifiedAt),
|
||||
Mod(2, seconds: 10, prior: 1.5, HistoryModificationType.Replace, "bob", modifiedAt.AddMinutes(1)),
|
||||
Mod(3, seconds: 20, prior: 7.0, HistoryModificationType.Delete, "carol", modifiedAt.AddMinutes(2)),
|
||||
};
|
||||
var (host, nm, node, src) = await SetupAsync(trail);
|
||||
|
||||
var r = Read(nm, node, Epoch, Epoch.AddHours(1), max: 100, cp: null);
|
||||
|
||||
r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||
r.Result.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||
r.Cp.ShouldBeNull();
|
||||
src.LastTag.ShouldBe(Tag);
|
||||
src.LastStart.ShouldBe(Epoch);
|
||||
src.LastEnd.ShouldBe(Epoch.AddHours(1));
|
||||
src.LastMax.ShouldBe(100u);
|
||||
src.LastCursor.ShouldBeNull();
|
||||
|
||||
var data = r.Data!;
|
||||
data.DataValues.Count.ShouldBe(3);
|
||||
data.ModificationInfos.Count.ShouldBe(3);
|
||||
data.ModificationInfos.Select(m => m.UpdateType)
|
||||
.ShouldBe([HistoryUpdateType.Insert, HistoryUpdateType.Replace, HistoryUpdateType.Delete]);
|
||||
data.ModificationInfos.Select(m => m.UserName).ShouldBe(["alice", "bob", "carol"]);
|
||||
data.ModificationInfos.Select(m => m.ModificationTime)
|
||||
.ShouldBe([modifiedAt, modifiedAt.AddMinutes(1), modifiedAt.AddMinutes(2)]);
|
||||
data.DataValues[0].Value.ShouldBeNull();
|
||||
data.DataValues[0].StatusCode.Code.ShouldBe(StatusCodes.BadNoData);
|
||||
data.DataValues[1].Value.ShouldBe(1.5);
|
||||
data.DataValues[1].SourceTimestamp.ShouldBe(Epoch.AddSeconds(10));
|
||||
data.DataValues[2].Value.ShouldBe(7.0);
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_empty_trail_is_GoodNoData()
|
||||
{
|
||||
var (host, nm, node, _) = await SetupAsync([]);
|
||||
var r = Read(nm, node, Epoch, Epoch.AddHours(1), max: 10, cp: null);
|
||||
r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||
r.Result.StatusCode.Code.ShouldBe(StatusCodes.GoodNoData);
|
||||
r.Data!.ModificationInfos.ShouldBeEmpty();
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Modified_reads_page_exactly_once_in_order_even_through_same_time_runs()
|
||||
{
|
||||
// 30 modifications; ids 5..9 all share one source time, so a page boundary cuts through the run.
|
||||
var trail = Enumerable.Range(1, 30)
|
||||
.Select(i => Mod(i, seconds: i is >= 5 and <= 9 ? 5 : i, prior: i, HistoryModificationType.Replace, "u", Epoch))
|
||||
.ToList();
|
||||
var (host, nm, node, src) = await SetupAsync(trail);
|
||||
|
||||
var ids = Drain(nm, node, Epoch, Epoch.AddHours(1), max: 4);
|
||||
|
||||
ids.ShouldBe(src.Ordered(reverse: false).Select(m => m.ModifiedId));
|
||||
ids.Count.ShouldBe(30);
|
||||
src.Calls.ShouldBe(8); // 7 full pages + a 2-value tail; the exact HasMore flag means no trailing empty page
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_reverse_modified_read_pages_descending()
|
||||
{
|
||||
var trail = Enumerable.Range(1, 11).Select(i => Mod(i, seconds: i / 2, prior: i, HistoryModificationType.Delete, "u", Epoch)).ToList();
|
||||
var (host, nm, node, src) = await SetupAsync(trail);
|
||||
|
||||
var ids = Drain(nm, node, Epoch.AddHours(1), Epoch.AddSeconds(-1), max: 3);
|
||||
|
||||
ids.ShouldBe(src.Ordered(reverse: true).Select(m => m.ModifiedId));
|
||||
ids.First().ShouldBe(11);
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_open_ended_forward_modified_read_pages_to_the_end()
|
||||
{
|
||||
var trail = Enumerable.Range(1, 9).Select(i => Mod(i, seconds: i, prior: i, HistoryModificationType.Update, "u", Epoch)).ToList();
|
||||
var (host, nm, node, src) = await SetupAsync(trail);
|
||||
|
||||
Drain(nm, node, Epoch, DateTime.MinValue, max: 4).ShouldBe(src.Ordered(reverse: false).Select(m => m.ModifiedId));
|
||||
src.LastEnd.ShouldBe(new DateTime(DateTime.MaxValue.Ticks, DateTimeKind.Utc));
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_unbounded_modified_read_never_pages()
|
||||
{
|
||||
var trail = Enumerable.Range(1, 50).Select(i => Mod(i, seconds: i, prior: i, HistoryModificationType.Update, "u", Epoch)).ToList();
|
||||
var (host, nm, node, _) = await SetupAsync(trail);
|
||||
var r = Read(nm, node, Epoch, Epoch.AddHours(1), max: 0, cp: null);
|
||||
r.Data!.DataValues.Count.ShouldBe(50);
|
||||
r.Cp.ShouldBeNull();
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Cursors_are_bound_to_the_read_kind_and_the_node()
|
||||
{
|
||||
var trail = Enumerable.Range(1, 10).Select(i => Mod(i, seconds: i, prior: i, HistoryModificationType.Update, "u", Epoch)).ToList();
|
||||
var (host, nm, node, _) = await SetupAsync(trail);
|
||||
|
||||
// A RAW cursor for the same tag is not a modified cursor.
|
||||
var raw = nm.HistoryContinuationStore.Save(null, new HistoryContinuationState(Tag, Epoch, Epoch.AddDays(1), 0, 4));
|
||||
Read(nm, node, Epoch, Epoch.AddHours(1), max: 4, cp: raw).Error.StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadContinuationPointInvalid);
|
||||
|
||||
// A MODIFIED cursor presented to a raw read is invalid too.
|
||||
var p1 = Read(nm, node, Epoch, Epoch.AddHours(1), max: 4, cp: null);
|
||||
p1.Cp.ShouldNotBeNull();
|
||||
Read(nm, node, Epoch, Epoch.AddHours(1), max: 4, cp: p1.Cp, modified: false).Error.StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadContinuationPointInvalid);
|
||||
|
||||
// A modified cursor issued for one node is invalid on another node's tag.
|
||||
var p2 = Read(nm, node, Epoch, Epoch.AddHours(1), max: 4, cp: null);
|
||||
nm.EnsureVariable("eq-1/other", null, "Other", "Double", false, historianTagname: "WW.Other", realm: AddressSpaceRealm.Uns);
|
||||
var other = nm.TryGetVariable("eq-1/other")!.NodeId;
|
||||
Read(nm, other, Epoch, Epoch.AddHours(1), max: 4, cp: p2.Cp).Error.StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadContinuationPointInvalid);
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_backend_failure_is_isolated_to_its_node()
|
||||
{
|
||||
var (host, nm, node, src) = await SetupAsync([Mod(1, 1, 1, HistoryModificationType.Update, "u", Epoch)]);
|
||||
src.Throw = new InvalidOperationException("boom");
|
||||
var r = Read(nm, node, Epoch, Epoch.AddHours(1), max: 4, cp: null);
|
||||
r.Error.StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
// --- helpers ---------------------------------------------------------------------------------
|
||||
|
||||
private static HistoryModifiedValue Mod(
|
||||
long id, int seconds, double? prior, HistoryModificationType type, string user, DateTime modifiedAt)
|
||||
{
|
||||
var t = Epoch.AddSeconds(seconds);
|
||||
var value = prior is null
|
||||
? new DataValueSnapshot(null, StatusCodes.BadNoData, t, modifiedAt)
|
||||
: new DataValueSnapshot(prior.Value, StatusCodes.Good, t, t);
|
||||
return new HistoryModifiedValue(value, modifiedAt, type, user, id);
|
||||
}
|
||||
|
||||
private static List<long> Drain(OtOpcUaNodeManager nm, NodeId node, DateTime start, DateTime end, uint max)
|
||||
{
|
||||
var ids = new List<long>();
|
||||
byte[]? cp = null;
|
||||
var pages = 0;
|
||||
do
|
||||
{
|
||||
var r = Read(nm, node, start, end, max, cp);
|
||||
r.Error.StatusCode.Code.ShouldBe(StatusCodes.Good, r.Error.ToString());
|
||||
ids.AddRange(r.Ids);
|
||||
cp = r.Cp;
|
||||
(++pages).ShouldBeLessThan(60, "paging must terminate");
|
||||
}
|
||||
while (cp is not null);
|
||||
return ids;
|
||||
}
|
||||
|
||||
private static (ServiceResult Error, SdkHistoryReadResult Result, HistoryModifiedData? Data, byte[]? Cp, List<long> Ids) Read(
|
||||
OtOpcUaNodeManager nm, NodeId node, DateTime start, DateTime end, uint max, byte[]? cp, bool modified = true)
|
||||
{
|
||||
var context = new OperationContext(new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
|
||||
var results = new List<SdkHistoryReadResult> { null! };
|
||||
var errors = new List<ServiceResult> { null! };
|
||||
nm.HistoryRead(context,
|
||||
new ReadRawModifiedDetails { StartTime = start, EndTime = end, NumValuesPerNode = max, IsReadModified = modified },
|
||||
TimestampsToReturn.Both, false,
|
||||
new List<HistoryReadValueId> { new() { NodeId = node, ContinuationPoint = cp } }, results, errors);
|
||||
|
||||
HistoryModifiedData? data = null;
|
||||
var ids = new List<long>();
|
||||
if (results[0]?.HistoryData is { } ho && !ExtensionObject.IsNull(ho)
|
||||
&& ExtensionObject.ToEncodeable(ho) is HistoryModifiedData hm)
|
||||
{
|
||||
data = hm;
|
||||
// The fake stores each modification's id in its (unique) server timestamp offset — see SqlLikeTrail.
|
||||
ids.AddRange(hm.DataValues.Select(v => SqlLikeTrail.IdOf(v)));
|
||||
}
|
||||
|
||||
return (errors[0], results[0], data, results[0]?.ContinuationPoint, ids);
|
||||
}
|
||||
|
||||
private async Task<(OpcUaApplicationHost Host, OtOpcUaNodeManager Nm, NodeId Node, SqlLikeTrail Src)> SetupAsync(
|
||||
List<HistoryModifiedValue> trail)
|
||||
{
|
||||
var started = await OpcUaHostStart.StartAsync(
|
||||
port => new OpcUaApplicationHost(
|
||||
new OpcUaApplicationHostOptions
|
||||
{
|
||||
ApplicationName = "OtOpcUa.ModifiedHistoryTest",
|
||||
ApplicationUri = $"urn:OtOpcUa.ModifiedHistoryTest:{Guid.NewGuid():N}",
|
||||
OpcUaPort = port,
|
||||
PublicHostname = "localhost",
|
||||
PkiStoreRoot = _pkiRoot,
|
||||
},
|
||||
NullLogger<OpcUaApplicationHost>.Instance),
|
||||
() => new OtOpcUaSdkServer(), Ct);
|
||||
var nm = started.Server.NodeManager!;
|
||||
nm.HistoryContinuationStore = new InMemoryHistoryContinuationStore();
|
||||
var src = new SqlLikeTrail(trail);
|
||||
nm.HistorianDataSource = src;
|
||||
nm.EnsureVariable("eq-1/mod", null, "Mod", "Double", false, historianTagname: Tag, realm: AddressSpaceRealm.Uns);
|
||||
return (started.Host, nm, nm.TryGetVariable("eq-1/mod")!.NodeId, src);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Behaves like the SqlServer/QuestDB modified reads: ordered by <c>(SourceTimeUtc, ModifiedId)</c> (descending
|
||||
/// when reversed), half-open window, exclusive cursor, cap + exact <see cref="HistoryModifiedReadResult.HasMore"/>.
|
||||
/// To let the test recover each projected value's id, the server timestamp handed out is
|
||||
/// <c>Epoch + id ticks</c> (an otherwise unused field in these tests).
|
||||
/// </summary>
|
||||
private sealed class SqlLikeTrail(List<HistoryModifiedValue> trail) : IHistorianDataSource
|
||||
{
|
||||
public string? LastTag { get; private set; }
|
||||
public DateTime LastStart { get; private set; }
|
||||
public DateTime LastEnd { get; private set; }
|
||||
public uint LastMax { get; private set; }
|
||||
public HistoryModifiedCursor? LastCursor { get; private set; }
|
||||
public int Calls { get; private set; }
|
||||
public Exception? Throw { get; set; }
|
||||
|
||||
public static long IdOf(DataValue v) => v.ServerTimestamp.Ticks - Epoch.Ticks;
|
||||
|
||||
public IEnumerable<HistoryModifiedValue> Ordered(bool reverse) => reverse
|
||||
? trail.OrderByDescending(m => m.Value.SourceTimestampUtc).ThenByDescending(m => m.ModifiedId)
|
||||
: trail.OrderBy(m => m.Value.SourceTimestampUtc).ThenBy(m => m.ModifiedId);
|
||||
|
||||
public async Task<HistoryModifiedReadResult> ReadModifiedAsync(
|
||||
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValues, HistoryModifiedCursor? resumeAfter,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
await Task.Delay(1, cancellationToken).ConfigureAwait(false);
|
||||
if (Throw is not null) throw Throw;
|
||||
Calls++;
|
||||
(LastTag, LastStart, LastEnd, LastMax, LastCursor) = (fullReference, startUtc, endUtc, maxValues, resumeAfter);
|
||||
var reverse = startUtc > endUtc;
|
||||
var (lo, hi) = reverse ? (endUtc, startUtc) : (startUtc, endUtc);
|
||||
var rows = Ordered(reverse).Where(m =>
|
||||
{
|
||||
var t = m.Value.SourceTimestampUtc!.Value;
|
||||
var inWindow = startUtc == endUtc ? t == lo : reverse ? t > lo && t <= hi : t >= lo && t < hi;
|
||||
if (!inWindow) return false;
|
||||
if (resumeAfter is null) return true;
|
||||
var c = t.CompareTo(resumeAfter.SourceTimeUtc);
|
||||
return reverse ? c < 0 || (c == 0 && m.ModifiedId < resumeAfter.ModifiedId)
|
||||
: c > 0 || (c == 0 && m.ModifiedId > resumeAfter.ModifiedId);
|
||||
}).Select(m => m with { Value = m.Value with { ServerTimestampUtc = Epoch.AddTicks(m.ModifiedId) } }).ToList();
|
||||
var take = maxValues == 0 ? rows.Count : (int)Math.Min(maxValues, (uint)rows.Count);
|
||||
return new HistoryModifiedReadResult(rows.Take(take).ToList(), rows.Count > take);
|
||||
}
|
||||
|
||||
public Task<HistorianRead> ReadRawAsync(string r, DateTime s, DateTime e, uint m, CancellationToken ct) =>
|
||||
NullHistorianDataSource.Instance.ReadRawAsync(r, s, e, m, ct);
|
||||
public Task<HistorianRead> ReadProcessedAsync(
|
||||
string r, DateTime s, DateTime e, TimeSpan i, HistoryAggregateType a, CancellationToken ct) =>
|
||||
NullHistorianDataSource.Instance.ReadProcessedAsync(r, s, e, i, a, ct);
|
||||
public Task<HistorianRead> ReadAtTimeAsync(string r, IReadOnlyList<DateTime> t, CancellationToken ct) =>
|
||||
NullHistorianDataSource.Instance.ReadAtTimeAsync(r, t, ct);
|
||||
public Task<HistoricalEventsResult> ReadEventsAsync(string? n, DateTime s, DateTime e, int m, CancellationToken ct) =>
|
||||
NullHistorianDataSource.Instance.ReadEventsAsync(n, s, e, m, ct);
|
||||
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
|
||||
public void Dispose() { }
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (Directory.Exists(_pkiRoot))
|
||||
{
|
||||
try { Directory.Delete(_pkiRoot, recursive: true); } catch { /* best effort */ }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -175,10 +175,13 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
/// <summary>Raw with IsReadModified=true ⇒ BadHistoryOperationUnsupported (we don't serve modified
|
||||
/// history); the source is never invoked.</summary>
|
||||
/// <summary>
|
||||
/// #668: Raw with IsReadModified=true on a backend that keeps no modification trail (the recording fake has no
|
||||
/// ReadModifiedAsync override, like Gateway) ⇒ BadHistoryOperationUnsupported, and the RAW read is never
|
||||
/// invoked in its place.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Raw_read_modified_yields_BadHistoryOperationUnsupported()
|
||||
public async Task Raw_read_modified_on_a_backend_without_a_trail_yields_BadHistoryOperationUnsupported()
|
||||
{
|
||||
var (host, server) = await BootAsync();
|
||||
var nm = server.NodeManager!;
|
||||
@@ -197,10 +200,16 @@ public sealed class NodeManagerHistoryReadTests : IDisposable
|
||||
IsReadModified = true,
|
||||
};
|
||||
|
||||
var (_, errors) = InvokeHistoryRead(server, nm, details, nodeId);
|
||||
var (results, errors) = InvokeHistoryRead(server, nm, details, nodeId);
|
||||
|
||||
errors[0].StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||
fake.LastCall.ShouldBeNull();
|
||||
results[0].StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||
fake.LastCall.ShouldBeNull(); // the raw read is never served in place of a modified one
|
||||
|
||||
// The Null default (no historian configured) answers the same.
|
||||
nm.HistorianDataSource = NullHistorianDataSource.Instance;
|
||||
InvokeHistoryRead(server, nm, details, nodeId).Item2[0].StatusCode.Code
|
||||
.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||
|
||||
await host.DisposeAsync();
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user