Merge branch 'master' into fix/832-docker-dev-batch
This commit is contained in:
@@ -39,6 +39,7 @@ All commands accept these options:
|
||||
| `-P` / `--password` | Password for `UserName` token authentication |
|
||||
| `-S` / `--security` | Transport security mode: `none`, `sign`, `encrypt`, `signandencrypt` (default: `none`) |
|
||||
| `-F` / `--failover-urls` | Comma-separated failover endpoint URLs for redundancy |
|
||||
| `--validate-host-name` | Require the server certificate to list the `--url` host (#816). Only with `-S sign`/`encrypt` — ignored for `-S none` (the SDK would otherwise check the GetEndpoints certificate even on an unsecured channel); off by default (the CLI's long-standing behaviour). The CLI auto-accepts untrusted certificates, so this checks a misconfiguration, not trust. Leave off behind NAT / when connecting by an IP the certificate omits / for a zone-qualified IPv6 host |
|
||||
| `--verbose` | Enable debug-level Serilog console logging (default: warning) |
|
||||
|
||||
**`--security none` does not fall back.** Endpoint selection picks the server's `None`-policy endpoint and
|
||||
|
||||
@@ -388,7 +388,7 @@ The Galaxy/MxAccess connection settings are **not an `appsettings` section.** Th
|
||||
|
||||
| Record | Key fields (default) | Meaning |
|
||||
|---|---|---|
|
||||
| `GalaxyGatewayOptions` (`Gateway`) | `Endpoint`; `ApiKeySecretRef`; `UseTls` (`true`); `CaCertificatePath` (`null`); `ConnectTimeoutSeconds` (`10`); `DefaultCallTimeoutSeconds` (`30`); `StreamTimeoutSeconds` (`0` = unlimited) | mxaccessgw gateway connection. `ApiKeySecretRef` supports `env:NAME` / `file:PATH` / `dev:KEY` / literal forms (resolved at `InitializeAsync`); prefer `env:`/`file:` in production. Never store a cleartext key. |
|
||||
| `GalaxyGatewayOptions` (`Gateway`) | `Endpoint`; `ApiKeySecretRef`; `UseTls` (`true`); `CaCertificatePath` (`null`); `ConnectTimeoutSeconds` (`10`); `DefaultCallTimeoutSeconds` (`30`); `StreamTimeoutSeconds` (`0` = unlimited) | mxaccessgw gateway connection. `ApiKeySecretRef` supports `secret:galaxy/<name>` (or `secret:otopcua/galaxy/<name>`) / `env:OTOPCUA_GALAXY_<NAME>` (or legacy `env:GALAXY_MXGW_<NAME>`) / `dev:KEY` / literal forms (resolved at `InitializeAsync`); prefer `secret:` in production. Any other `secret:`/`env:` reference and every `file:` reference is refused at save, deploy (`CredentialReferenceOutOfScope`), Test Connect, browse and resolve — #815, see `docs/security.md` §"Credential references are namespaced per driver". Never store a cleartext key. |
|
||||
| `GalaxyMxAccessOptions` (`MxAccess`) | `ClientName`; `PublishingIntervalMs` (`1000`); `WriteUserId` (`0` = anonymous); `EventPumpChannelCapacity` (`50000`) | MXAccess client identity + tuning. `ClientName` **must be unique per OtOpcUa instance** (redundancy pairs enforce this). |
|
||||
| `GalaxyRepositoryOptions` (`Repository`) | `DiscoverPageSize` (`5000`); `WatchDeployEvents` (`true`) | Galaxy Repository browse paging + deploy-event watching. |
|
||||
| `GalaxyReconnectOptions` (`Reconnect`) | `InitialBackoffMs` (`500`); `MaxBackoffMs` (`30000`); `ReplayOnSessionLost` (`true`) | In-driver reconnect-supervisor backoff. |
|
||||
|
||||
+45
-18
@@ -108,7 +108,7 @@ consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `A
|
||||
| Value kinds recorded | `Double`, `Int64`, `Boolean` (numeric analog only) | all: `Double`, `Int64`, `Boolean`, `String`, `DateTime`, plus `Empty` (a Bad sample with no value) | same as `SqlServer` |
|
||||
| Status code stored | Sent as DA quality but **discarded** by the gateway (C-002); AVEVA stores every sample Good | Stored bit-for-bit (`StatusCode` int column) | Stored bit-for-bit (unsigned, in a `LONG`) |
|
||||
| Comms-loss / driver-Bad publishes | Dropped, counted in `DroppedUnsupportedKind` | Stored as an `Empty` outage-marker sample carrying the Bad code (see [below](#typed-values-real-status-and-outage-markers)) | Stored as an `Empty` outage-marker sample, as on `SqlServer` |
|
||||
| Processed aggregates | **Ten** by default: the per-sample ones (Average, Count, Minimum, Maximum, Range, Start, End, Delta, Min/MaxActualTime) evaluated from raw reads with the SQL backends' evaluator — identical results (#681). TimeAverage and Total from AVEVA's native modes **only with `ServerHistorian:Gateway:EnableNativeTimeWeighted=true`** (default off: Good-status wrong values until historiangw#27). TimeAverage2 / Total2 / Interpolative / PercentGood / PercentBad: never | The seventeen in the [aggregate table](#aggregates-processed-reads) (incl. Part 13 time-weighted, #671) | The same seventeen, identical results |
|
||||
| Processed aggregates | **Ten** by default: the per-sample ones (Average, Count, Minimum, Maximum, Range, Start, End, Delta, Min/MaxActualTime) evaluated from raw reads with the SQL backends' evaluator — identical results (#681). TimeAverage and Total from AVEVA's native modes **only with `ServerHistorian:Gateway:EnableNativeTimeWeighted=true`** (default off: Good-status wrong values until historiangw#27). TimeAverage2 / Total2 / Interpolative / PercentGood / PercentBad / DurationGood / DurationBad: never | The nineteen in the [aggregate table](#aggregates-processed-reads) (incl. Part 13 time-weighted, #671, and DurationGood/DurationBad, #820) | The same nineteen, identical results |
|
||||
| Reverse / end-only Raw and Events reads | Emulated: walked back from bounded forward reads by the server (`ReverseRawReader` / `ReverseEventsReader`) — correct, but many round trips on a sparse tag | Native newest-first | Native newest-first |
|
||||
| Capped Events read | The cap is applied client-side (the gateway RPC has no max-events field); the stream past it is drained up to `Gateway:EventDrainBudget` rows | `TOP` | `LIMIT` |
|
||||
| Retention | AVEVA Historian's own | `SqlServer:RetentionDays` purge (opt-in) | `QuestDb:RetentionDays` TTL (opt-in) |
|
||||
@@ -192,7 +192,7 @@ Processed read is `BadAggregateNotSupported` and no history aggregate functions
|
||||
| `EventReadHorizon` | TimeSpan | `00:00:00` | (#694, all backends, **opt-in, lossy**.) `00:00:00` = no horizon: every HistoryReadEvents read is exact to its own window. A positive span raises the earlier bound of every fresh events window to `now − EventReadHorizon` (the floor instant itself included), in **both** directions — a window wholly older is `GoodNoData` and is not sent; a page-through keeps the first page's floor. It exists for the Gateway, where a walked reverse / end-only read proves "nothing older" down to the Unix epoch (~3.8 s live; an unfiltered one can hit the gateway's 30 s SQL timeout) and a forward read from a far-past `StartTime` scans the same range. ⚠️ While set, events older than the horizon are **invisible** to HistoryReadEvents. Raw/Processed/AtTime are unaffected. Negative ⇒ treated as none (a `Validate()` warning). See [Continuation-point paging (Events)](#continuation-point-paging-events-669). |
|
||||
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
|
||||
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead (and HistoryUpdate) batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` (#671), and the `Gateway` per-sample aggregates (#681 — every per-sample read there streams raw samples, so the cap applies to the window's raw rows; see [Aggregate set](#aggregate-set-and-badaggregatenotsupported-285)). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped on the SQL backends (pushed down). 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). `Gateway` (#688): the row budget it asks the gateway for (`cap + 18`) is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is at or under it, so the default works against a gateway with `AllowStreamingBeyondCeiling=false`; against a gateway with a *lower* ceiling the refused budget is retried once at the ceiling the gateway names (#691.3), so no change is needed (`ceiling − 18` only saves that round-trip). |
|
||||
| `MaxTimeWeightedRawRows` | int | `1000000` | `SqlServer` / `QuestDb` (#671), and the `Gateway` per-sample aggregates (#681 — every per-sample read there streams raw samples, so the cap applies to the window's raw rows; see [Aggregate set](#aggregate-set-and-badaggregatenotsupported-285)). The most raw in-window samples one [time-weighted aggregate](#time-weighted-aggregates-part-13-671) read (TimeAverage, TimeAverage2, **Total**, Total2, Interpolative, and — since #680 — **PercentGood/PercentBad**, since #820 **DurationGood/DurationBad**) may stream into the server-side evaluator — that family is **not** pushed down to SQL. The window query is capped at this + 1 rows; one row past the cap fails **that node** with `BadTooManyOperations` (not counted as a backend fault) instead of holding a pooled connection until `HistoryReadDeadline` and returning `BadTimeout`. Bound rows outside the window do not count; per-sample aggregates are not capped on the SQL backends (pushed down). 1 000 000 ≈ 11.5 days of a 1 Hz tag. Non-positive = unbounded (a `Validate()` warning). `Gateway` (#688): the row budget it asks the gateway for (`cap + 18`) is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is at or under it, so the default works against a gateway with `AllowStreamingBeyondCeiling=false`; against a gateway with a *lower* ceiling the refused budget is retried once at the ceiling the gateway names (#691.3), so no change is needed (`ceiling − 18` only saves that round-trip). |
|
||||
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead (and HistoryUpdate) batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||
| `HistoryUpdateMaxFutureSkew` | TimeSpan | `1.00:00:00` | #674. How far past the server clock a HistoryUpdate (Insert / Replace / Update) item's `SourceTimestamp` may lie; an item beyond `now + skew` is refused **`BadOutOfRange`** by the node manager before any backend sees it (every backend). Guards against a far-future typo (a 2062 timestamp). The live recorder is not bounded. Non-positive disables the bound (no warning — a deliberate choice). See [HistoryUpdate](#historyupdate). |
|
||||
|
||||
@@ -608,7 +608,7 @@ The server supports all four OPC UA HistoryRead variants:
|
||||
| Variant | Node type | CLI `--aggregate` |
|
||||
|---|---|---|
|
||||
| **Raw** | Historized variable | (omit `--aggregate`) |
|
||||
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`, `StandardDeviation` (the CLI's own list; `SqlServer` / `QuestDb` serve seventeen aggregates — see the [aggregate table](#aggregates-processed-reads)) |
|
||||
| **Processed** | Historized variable | `Average`, `Minimum`, `Maximum`, `Count`, `Start`, `End`, `StandardDeviation` (the CLI's own list; `SqlServer` / `QuestDb` serve nineteen aggregates — see the [aggregate table](#aggregates-processed-reads)) |
|
||||
| **AtTime** | Historized variable | n/a (client supplies exact timestamps) |
|
||||
| **Events** | Equipment folder (event notifier) | `--events` (+ `--fields`, `--of-type`; #776) |
|
||||
| **Modified** (Raw with `IsReadModified = true`, #668) | Historized variable | `--modified` (see [Modified-value history](#modified-value-history-part-11)) |
|
||||
@@ -1312,7 +1312,7 @@ field (`tail + 43`) its own reverse-engineering notes still mark unverified (His
|
||||
> `Integral`) — ⚠️ **and are refused `BadAggregateNotSupported` unless `ServerHistorian:Gateway:EnableNativeTimeWeighted` is
|
||||
> `true`** (default `false`, until historiangw#27 is live-verified; the default set is the ten raw-evaluated aggregates,
|
||||
> `GatewayHistorianDataSource.RawEvaluatedAggregates`) — now stamped with the interval start (was its end) and carrying the Calculated bit; `TimeAverage2`, `Total2`,
|
||||
> `Interpolative`, `PercentGood` and `PercentBad` remain `BadAggregateNotSupported` on Gateway. The most the backend can serve is
|
||||
> `Interpolative`, `PercentGood`, `PercentBad`, `DurationGood` and `DurationBad` remain `BadAggregateNotSupported` on Gateway. The most the backend can serve is
|
||||
> `GatewayHistorianDataSource.ServedAggregates` (12); an instance serves and advertises `SupportedAggregates` — those 12 with the
|
||||
> flag on, the 10 raw-evaluated ones without it.
|
||||
|
||||
@@ -1378,8 +1378,8 @@ and one ending at `now + 1 day` are accepted and cheap; (4) the gateway's ceilin
|
||||
Min/MaxActualTime, Start/EndBound, DeltaBounds, DurationGood/Bad, PercentGood/Bad, WorstQuality,
|
||||
NumberOfTransitions, AnnotationCount, StandardDeviationPopulation, VariancePopulation); `AggregateFunctionMap`
|
||||
(`OpcUaServer`) maps each standard aggregate NodeId (`TimeAverage2` and `Total2` were appended for #671). The **`SqlServer`** and
|
||||
**`QuestDb`** backends serve the seventeen listed in the [aggregate table](#aggregates-processed-reads); everything else
|
||||
(e.g. `StartBound`, `DurationGood`, `StandardDeviationPopulation`) is `BadAggregateNotSupported` for that node. The **HistorianGateway** backend serves twelve (#681 —
|
||||
**`QuestDb`** backends serve the nineteen listed in the [aggregate table](#aggregates-processed-reads); everything else
|
||||
(e.g. `StartBound`, `WorstQuality`, `StandardDeviationPopulation`) is `BadAggregateNotSupported` for that node. The **HistorianGateway** backend serves twelve (#681 —
|
||||
`GatewayHistorianDataSource.ServedAggregates`): the ten per-sample aggregates from raw reads, plus TimeAverage and Total
|
||||
through AVEVA retrieval modes (`AggregateModeMapper`); for any other member it throws
|
||||
`HistoryAggregateNotSupportedException` without calling the gateway (and since #682 the server refuses it up front), which the HistoryRead path turns into
|
||||
@@ -1407,7 +1407,7 @@ aggregate-function objects. A client choosing a history aggregate should read **
|
||||
| `ServerHistorian:Backend` | i=11201 (history) | i=2997 (server) | Declared by |
|
||||
|---|---|---|---|
|
||||
| `Gateway` | 12 — the ten per-sample aggregates (raw-evaluated, #681) + TimeAverage, Total (native AVEVA modes) | all 37 | `GatewayHistorianDataSource.ServedAggregates` (the per-sample families of `AggregateFamilies` ∪ `AggregateModeMapper.NativeAggregates`, derived from its mapping switch) |
|
||||
| `SqlServer`, `QuestDb` | 17 — the [aggregate table](#aggregates-processed-reads) | all 37 | `AggregateFamilies.Served` (Sql.Common; every member with a family) |
|
||||
| `SqlServer`, `QuestDb` | 19 — the [aggregate table](#aggregates-processed-reads) (DurationGood/DurationBad added by #820) | all 37 | `AggregateFamilies.Served` (Sql.Common; every member with a family) |
|
||||
| historian disabled (Null source) | **none** | all 37 | `NullHistorianDataSource` |
|
||||
|
||||
⚠️ **Client-visible: with no historian configured a processed HistoryRead is now `BadAggregateNotSupported`, not
|
||||
@@ -1573,9 +1573,10 @@ point (see [Continuation-point paging (Events)](#continuation-point-paging-event
|
||||
|
||||
> **Two kinds.** The *per-sample* aggregates (Average, Minimum, Maximum, Count, Range, Start/End/Delta, Min/MaxActualTime) are
|
||||
> simple per-sample definitions — no time-weighting, no interval-boundary values — pushed down to SQL. The *time-weighted* aggregates
|
||||
> (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671 — and the duration-based PercentGood/PercentBad since #680) follow
|
||||
> (TimeAverage, TimeAverage2, Total, Total2, Interpolative; #671 — and the duration-based PercentGood/PercentBad since #680,
|
||||
> DurationGood/DurationBad since #820) follow
|
||||
> OPC UA Part 13 with bounding values — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671). Any `HistoryAggregateType` not in the tables throws
|
||||
> `HistoryAggregateNotSupportedException` ⇒ `BadAggregateNotSupported` for that node (e.g. `StartBound`, `DurationGood`,
|
||||
> `HistoryAggregateNotSupportedException` ⇒ `BadAggregateNotSupported` for that node (e.g. `StartBound`, `WorstQuality`,
|
||||
> `StandardDeviationPopulation`).
|
||||
>
|
||||
> ⚠️ **Release note / migration (#671): `Total` changed meaning on the `SqlServer` and `QuestDb` backends.** It was the plain `SUM` of
|
||||
@@ -1716,12 +1717,12 @@ Status and edge rules:
|
||||
- An interval with samples but **no Good numeric value** (only Uncertain / Bad) ⇒ `BadNoData` for `Average`/`Minimum`/`Maximum`/`Range`/`*ActualTime` (#684).
|
||||
- `Average` is `Good` when every sample in the bucket is Good, else `UncertainDataSubNormal` (value-based, the SDK's rule). `Minimum`/`Maximum`/`Range`/`*ActualTime` are `UncertainDataSubNormal` when the bucket holds a Bad sample or an Uncertain value beyond the reported Good extreme (Part 13 Tables 57–62; strictly beyond), else `Good` (#684). `Count`'s status is value-based (all Good ⇒ Good, all Bad ⇒ Bad, else Uncertain); `Start`/`End` carry the raw sample's own status; `Delta` is Good unless the interval's first or last sample is not usable (then Uncertain) — #680.
|
||||
- `Count`, `Start`, `End` accept any value kind. `Average`/`Minimum`/`Maximum`/`Range`/`*ActualTime` on a bucket that holds a **Good** `String`/`DateTime` sample ⇒ `BadAggregateInvalidInputs` for that interval; an Uncertain one is skipped and makes `Minimum`/`Maximum`/`Range`/`*ActualTime` `UncertainDataSubNormal` (#684). `Delta` is refused by any usable non-numeric sample.
|
||||
- `PercentGood` / `PercentBad` are **duration-based since #680** — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671).
|
||||
- `PercentGood` / `PercentBad` are **duration-based since #680**, and `DurationGood` / `DurationBad` (#820) are the same region times — see [Time-weighted aggregates](#time-weighted-aggregates-part-13-671).
|
||||
- Everything except `Count` and `Start`/`End` is a `double`; `Int64` samples aggregate as doubles and `Boolean` as 0/1.
|
||||
|
||||
### Time-weighted aggregates (Part 13, #671)
|
||||
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2`, `Interpolative` and (since #680) `PercentGood` / `PercentBad`, evaluated by the
|
||||
`TimeAverage`, `TimeAverage2`, `Total`, `Total2`, `Interpolative`, (since #680) `PercentGood` / `PercentBad` and (since #820) `DurationGood` / `DurationBad`, evaluated by the
|
||||
shared `Driver.Historian.Sql.Common/TimeWeightedAggregator.cs` for **both** SQL backends (so `SqlServer` and `QuestDb` agree exactly).
|
||||
**This whole family — `Total` and the percentages included — is evaluated in C#, not pushed down to SQL** (see *Pushdown trade-off*
|
||||
and the raw-row cap below). Bucketing (edges, truncated interval, reverse anchoring, the 100 000-interval cap) is the same as above.
|
||||
@@ -1784,6 +1785,7 @@ samples (sloped / stepped) straight across an outage, and holds the last usable
|
||||
| `Total2` | ∫ simple-bounding signal over non-Bad time (value·s) | excluded |
|
||||
| `Interpolative` | interpolated-bounding value **at the interval start** (reverse read: see below) | interpolated across |
|
||||
| `PercentGood` / `PercentBad` | `100 ×` Good / Bad time ÷ interval duration (#680, Part 13 §5.4.3.31-34) — see below | counted as Bad time |
|
||||
| `DurationGood` / `DurationBad` | Good / Bad time of the interval in **milliseconds** (an OPC UA `Duration`; #820, Part 13 §5.4.3.31-32) — the same regions, see below | counted as Bad time |
|
||||
|
||||
**Status, `TimeAverage` / `Total`** (interpolated bounding, #680 — Part 13 "Uncertain if Bad values are skipped or Uncertain values
|
||||
are used", and the SDK's `AverageAggregateCalculator`):
|
||||
@@ -1822,6 +1824,14 @@ to the **end of its own interval** (Part 13 — the SDK gives it 1 ms) and later
|
||||
after it is Bad to the window end. An interval that ends before the first sample ⇒ `BadNoData`. Any value kind is accepted; the status
|
||||
is `Good` (plus Calculated / Partial). Value = `100 × region time ÷ interval duration`.
|
||||
|
||||
**`DurationGood` / `DurationBad`** (#820) are the same Good / Bad region times as `PercentGood` / `PercentBad` — every rule above
|
||||
(regions, simple bounds, time before the first sample = Bad, end of data, `BadNoData`, status, Partial) applies unchanged — reported as
|
||||
an OPC UA `Duration`, i.e. a `double` in **milliseconds** (the SDK's unit): `DurationGood = PercentGood × interval ms ÷ 100`. Uncertain
|
||||
time counts toward neither (Mantis 0006166). Before #820 both were `BadAggregateNotSupported` on the SQL backends. ⚠️ A **live**
|
||||
`AggregateFilter` subscription computes DurationGood/PercentGood with the frozen 1.5.378.182 calculators, which count Uncertain time as
|
||||
Good — they disagree with HistoryRead whenever Uncertain data is present (#809, owner decision pending). The Gateway backend still
|
||||
refuses both (`BadAggregateNotSupported`, not advertised in i=11201).
|
||||
|
||||
All carry the Calculated bit. A non-Bad result also carries the **Partial** bit (`0x4`, `AggregateBits.Partial`) — never on a Bad
|
||||
result — for: the **truncated interval** (shorter than the processing interval — the latest one forward, the earliest one reverse;
|
||||
Part 13 §5.3.3.2 Case 3 requires it — the SDK does not set it, a deliberate difference); the interval the tag's **data starts inside**
|
||||
@@ -1856,7 +1866,7 @@ value at **10:00**, not 09:00, and no point ever falls before the window. The in
|
||||
forward and reverse, and compare every
|
||||
interval's timestamp, value, status code and Partial bit. The per-sample comparison covers Average, Minimum, Maximum, Count, Range,
|
||||
Start, End, Delta and Minimum/MaximumActualTime; the time-weighted one TimeAverage, TimeAverage2, Total, Total2, Interpolative and
|
||||
PercentGood/Bad (#680), including datasets whose data ends inside the window and with an interval wholly inside an outage; reverse
|
||||
PercentGood/Bad (#680) and DurationGood/Bad (#820), including datasets whose data ends inside the window and with an interval wholly inside an outage; reverse
|
||||
reads over data that starts inside the window are compared too. Known differences that remain (tracked as
|
||||
#680) — documented and masked in the tests, deliberately not changed here because forward results were to stay as they are — are:
|
||||
an Uncertain value beyond the Good extreme makes Minimum/Maximum/Range/*ActualTime Uncertain here (Part 13), Good in the SDK
|
||||
@@ -1895,9 +1905,14 @@ And an interval holding a **Good String (or DateTime) sample** is `BadAggregateI
|
||||
string and serves the numeric samples (e.g. Average 4 Good); an Uncertain non-numeric sample refuses nothing on either side.
|
||||
One more, found by the #810 Uncertain conformance data (kept deliberately): over the interval the data **starts inside** when the
|
||||
**first sample is Uncertain**, the SDK's `TimeAverage`/`Total` integrate the uncovered span before it as a ramp from 0 (QP: `Total`
|
||||
188.96 for 163.96 over the covered 15 s); ours integrates only the covered part (Part 13, as with a Good first sample, where the two
|
||||
agree). Pinned by `QualityMixConformance.Time_weighted_values_over_partial_data_cover_only_the_data` on both SQL backends. The same
|
||||
suite feeds PercentGood/PercentBad (and Count, Average, TimeAverage(2), Total(2), Start/End/Delta, Interpolative) Good/Uncertain/Bad
|
||||
188.96 for 163.96 over the covered 15 s, and `TimeAverage` 9.448 = 188.96 ÷ 20 s for 10.931 = 163.96 ÷ 15 s); ours integrates only
|
||||
the covered part (Part 13, as with a Good first sample, where all agree). The same answer comes from the frozen calculators the server
|
||||
registers for **live** `AggregateFilter` items and from SDK 2.0.1's own stock calculators, in both read directions — pinned by
|
||||
`SdkAggregateParityTests.An_Uncertain_first_sample_in_a_late_start_interval_makes_the_SDK_integrate_the_time_before_the_data`
|
||||
(ours / frozen / stock 2.0.1, with a Good-first control) and `OpcUaServer.IntegrationTests/LiveAggregateLateStartFactsTests` (through
|
||||
the booted server's runtime `AggregateManager`), and on both SQL backends by `QualityMixConformance.Time_weighted_values_over_partial_data_cover_only_the_data`
|
||||
(forward) and `…_in_a_reverse_read` (#821). Whether live subscriptions should follow is the open #809/#821 owner decision. The same
|
||||
suite feeds PercentGood/PercentBad, DurationGood/DurationBad (and Count, Average, TimeAverage(2), Total(2), Start/End/Delta, Interpolative) Good/Uncertain/Bad
|
||||
mixes, all-Uncertain intervals (0 % Good **and** 0 % Bad, Mantis 0006166) and partial-data intervals, hand-computed and against the
|
||||
#796 oracle — the baseline calculator's Uncertain-as-Good PercentGood fails it.
|
||||
|
||||
@@ -1991,10 +2006,15 @@ round-trip, redelivery, raw / AtTime / aggregate / event reads, every update res
|
||||
run creates a throwaway `OtOpcUaHistorian_T_<guid>` database and drops it on dispose.
|
||||
|
||||
```bash
|
||||
export HISTORIAN_SQL_TEST_CONNECTION='Server=localhost,14330;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;Encrypt=False'
|
||||
# The shared always-on dev SQL Server on the Docker host (otopcua-mssql; the committed dev sa credential, see
|
||||
# docs/v2/dev-environment.md — not a secret):
|
||||
export HISTORIAN_SQL_TEST_CONNECTION='Server=10.100.0.35,14330;User Id=sa;Password=OtOpcUaDev_2026!;TrustServerCertificate=True;Encrypt=False'
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.SqlServer.IntegrationTests
|
||||
```
|
||||
|
||||
The local docker-dev rig's `sql` service also publishes `localhost,14330`, but with **its own** committed password
|
||||
(`OtOpcUa!Dev123`, `docker-dev/docker-compose.yml`) — the two are different instances; do not mix the host and the password (#822).
|
||||
|
||||
---
|
||||
|
||||
## QuestDB backend
|
||||
@@ -2006,7 +2026,7 @@ the spike that pinned every QuestDB-specific form: [plans/2026-10-10-questdb-spi
|
||||
`questdb/questdb:10.0.1`.
|
||||
|
||||
QuestDB serves every value kind (`Double`, `Int64`, `Boolean`, `String`, `DateTime`, `Empty`), stores the real status, records comms-loss
|
||||
outage markers, serves the seventeen aggregates (time-weighted ones through the same shared evaluator, so results are identical) and supports **HistoryUpdate** (soft delete), exactly like `SqlServer`. The same conformance
|
||||
outage markers, serves the nineteen aggregates (time-weighted ones through the same shared evaluator, so results are identical) and supports **HistoryUpdate** (soft delete), exactly like `SqlServer`. The same conformance
|
||||
suite (`tests/Drivers/Historian.Conformance`) runs against both backends; the differences below are the only ones.
|
||||
|
||||
### Configuration (`ServerHistorian:QuestDb`)
|
||||
@@ -2276,11 +2296,18 @@ tombstones and redelivery, the visibility wait, updater, reads, paging, and the
|
||||
unique throwaway prefix (`t_<guid>_`) and drops its tables afterwards. The QuestDB server allows 64 connections, so the fixture prunes idle pool connections.
|
||||
|
||||
```bash
|
||||
# The always-on shared fixture on the Docker host (#822; stack /opt/otopcua-questdb, infra/README.md §2a):
|
||||
export HISTORIAN_QUESTDB_TEST_CONNECTION='Host=10.100.0.35;Port=8812;Username=admin;Password=quest;Database=qdb;ServerCompatibilityMode=NoTypeLoading'
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests
|
||||
|
||||
# …or a local one from the docker-dev overlay (same image, PG wire on localhost:18812):
|
||||
docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml up -d --no-deps questdb
|
||||
export HISTORIAN_QUESTDB_TEST_CONNECTION='Host=localhost;Port=18812;Username=admin;Password=quest;Database=qdb;ServerCompatibilityMode=NoTypeLoading'
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests
|
||||
```
|
||||
|
||||
`admin` / `quest` is QuestDB's stock default — a throwaway fixture credential, not a secret. The fixture's compose file is
|
||||
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.IntegrationTests/Docker/docker-compose.yml` (label `project=lmxopcua`).
|
||||
|
||||
---
|
||||
|
||||
## HistoryUpdate
|
||||
|
||||
@@ -81,7 +81,7 @@ Project root files:
|
||||
|
||||
`DriverConfig` JSON binds to `Config/GalaxyDriverOptions.cs`. The four sections are:
|
||||
|
||||
- **`Gateway`** — endpoint, API key secret ref, TLS knobs, connect/call/stream timeouts. `StreamTimeoutSeconds = 0` keeps the long-lived `StreamEvents` RPC open for the driver's lifetime.
|
||||
- **`Gateway`** — endpoint, API key secret ref (`secret:galaxy/<name>` · `secret:otopcua/galaxy/<name>` · `env:OTOPCUA_GALAXY_<NAME>` · legacy `env:GALAXY_MXGW_<NAME>` · `dev:KEY` · literal; anything else, incl. `file:`, is refused — #815 (an already-deployed driver keeps resolving a pre-#815 ref on init and every gateway reconnect in deprecated warn-only mode until its next deploy), [`docs/security.md`](../security.md) §"Credential references are namespaced per driver"), TLS knobs, connect/call/stream timeouts. `StreamTimeoutSeconds = 0` keeps the long-lived `StreamEvents` RPC open for the driver's lifetime.
|
||||
- **`MxAccess`** — `ClientName` (must be unique per OtOpcUa instance — redundancy pairs enforce uniqueness at install time), `PublishingIntervalMs` (forwarded as `buffered_update_interval_ms` on subscribe), `WriteUserId` for ArchestrA secured-write, `EventPumpChannelCapacity` (default 50_000 — one second of headroom at 50k tags / 1Hz; tune via the `galaxy.events.dropped` metric).
|
||||
- **`Repository`** — `DiscoverPageSize`, `WatchDeployEvents`.
|
||||
- **`Reconnect`** — `InitialBackoffMs`, `MaxBackoffMs`, `ReplayOnSessionLost` (calls the gateway's `ReplaySubscriptions` RPC after reconnect rather than re-issuing subscribe-bulk for every tag).
|
||||
@@ -103,8 +103,10 @@ Galaxy password, so `GalaxyAckCredentialResolver` picks the account per acknowle
|
||||
not silently downgraded to the fallback.
|
||||
2. **The driver's fallback account** — `AlarmAck.FallbackUser` / `FallbackDomain` /
|
||||
`FallbackPasswordSecretRef` in `DriverConfig`, used by keys without an identity and by anonymous
|
||||
sessions. The password ref must be `secret:NAME` / `${secret:NAME}` (production), `env:NAME`,
|
||||
`file:PATH` or `dev:VALUE`; a bare literal is refused so a password never sits in DriverConfig.
|
||||
sessions. The password ref must be `secret:galaxy/NAME` / `secret:otopcua/galaxy/NAME` (or the `${secret:…}`
|
||||
spelling; production), `env:OTOPCUA_GALAXY_NAME` / `env:GALAXY_MXGW_NAME`, or `dev:VALUE`; a bare literal, a
|
||||
`file:` reference or a reference outside the Galaxy namespace is refused (#815) — the password is sent to the
|
||||
authored gateway endpoint, so it must not be able to name another subsystem's secret.
|
||||
3. **Neither** → the driver throws `DriverOperationRefusedException`; **nothing is sent to the
|
||||
gateway**. The refusal is excluded from the resilience breaker (it says nothing about gateway
|
||||
health). How the client learns of it (#760):
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Heidenhain LSV2 driver (read-only)
|
||||
|
||||
`DriverType = "Heidenhain"` (`DriverTypeNames.Heidenhain`) · projects `Driver.Heidenhain` (+ `.Contracts`) ·
|
||||
Gitea **#803**.
|
||||
Gitea **#803** (native alarms + rediscovery: **#831**; authoring UX: **#830**).
|
||||
|
||||
A native, **read-only** Heidenhain LSV2 client over TCP (default port **19000**) for iTNC/TNC controls. No
|
||||
Java/Python runtime and no vendor SDK (RemoTools/DNC) — a managed `System.Net.Sockets` client. It is a port of the
|
||||
@@ -21,9 +21,12 @@ Three independent layers, each sufficient on its own for its concern:
|
||||
manual-entry grid also fix the access level at Read — and again by the deploy gate `HeidenhainTagWritable`, so no
|
||||
node is ever advertised writable. A `"writable": true` TagConfig key is rejected by the tag reader and dropped by
|
||||
the typed editor. The same paths tie `Tag.DataType` to what the driver publishes (the catalog type for a fixed
|
||||
address — `Float64` ⇒ `Double` — or the binding's declared type): the modals default it from the address, the
|
||||
save path refuses a mismatch, and the deploy gate `HeidenhainTagDataTypeMismatch` catches anything older
|
||||
(one rule, `HeidenhainTagRules`).
|
||||
address — `Float64` ⇒ `Double` — or the binding's declared type; `Boolean` for an `ALARM:` condition): the tag
|
||||
modal and the manual-entry grid **lock** the access level and data type to the driver's values (#830 — the grid
|
||||
used to offer a choice it silently replaced on commit), the CSV import's staged review grid flags a ReadWrite or
|
||||
mistyped row **in the preview**, the save path refuses a mismatch, and the deploy gate
|
||||
`HeidenhainTagDataTypeMismatch` catches anything older. One rule (`HeidenhainTagRules`), reached by every AdminUI
|
||||
path through the driver-dispatched `RawTagRowRules` (no call site names a driver).
|
||||
3. **Payload-aware outbound allowlist** (`Lsv2ReadOnlyPolicy`). `Lsv2Codec.Frame` — the only frame builder — checks
|
||||
command **and** payload immediately before any byte is written. There is no raw-command API (`Lsv2Client`'s
|
||||
public surface is pinned by a reflection test) and no configuration knob that widens it.
|
||||
@@ -66,6 +69,7 @@ enums, so the numeric-enum form/factory trap cannot occur):
|
||||
| `Access.DataEnabled` | false | Opt-in DATA login, attempted only when a `DATA:` binding exists (iTNC only) |
|
||||
| `Access.InspectPassword` / `DncPassword` / `DataPassword` | — | **References only** (below) |
|
||||
| `Polling.StatusMs` / `CustomMs` / `SlowMs` | 1000 / 1000 / 10000 | each 100–3600000 |
|
||||
| `Polling.AlarmMs` | 2000 | 100–3600000; the alarm walk behind `ALARM:` conditions — only for a control carrying an `ALARM:` tag ([Alarm conditions](#alarm-conditions-alarm-gitea-831)) |
|
||||
| `Polling.MaxAlarms` | 64 | 1–256 |
|
||||
| `Polling.TimersEnabled` | **false** | Experimental selectors 30–34 |
|
||||
| `Reconnect.InitialDelayMs` / `MaxDelayMs` | 1000 / 30000 | doubling backoff, bounded |
|
||||
@@ -90,8 +94,13 @@ Host at their own machine. Anything outside the namespace is refused by **every*
|
||||
(`HeidenhainOptionsParser`, so the factory and the probe), the resolver **again at resolve time**
|
||||
(`HeidenhainCredentials`, fail-closed independently of earlier validation), the AdminUI **save path**
|
||||
(`RawTreeService` create/update driver + device refuse the config server-side via `DriverConfigSaveGate`, and the
|
||||
driver/device modals disable Save while it is invalid — so a literal never reaches the config database), and the
|
||||
deploy gate `HeidenhainCredentialPersisted`. Messages never echo the value.
|
||||
driver/device modals disable Save while it is invalid and show the refusal **once**, below the form — the typed
|
||||
forms report only ranges/endpoints, #830 — so a literal never reaches the config database), and the deploy gate
|
||||
`HeidenhainCredentialPersisted`. Messages never echo the value. Since #815 the parser and resolver are
|
||||
the shared `CredentialReferences` / `DriverCredentialPolicy` (`Core.Abstractions`) that Galaxy and OpcUaClient use
|
||||
too, and Test Connect / live browse also refuse an out-of-namespace reference before any probe or browser runs; a
|
||||
prefix in another casing (`SECRET:`) is refused, and `${secret:heidenhain/<name>}` is accepted as a spelling of
|
||||
`secret:heidenhain/<name>`.
|
||||
|
||||
Resolved **at each login**, never held in options or session state. Fail-closed: an absent secret or unset
|
||||
variable fails the login rather than sending the reference text. An unusable **INSPECT** credential fails the
|
||||
@@ -147,6 +156,70 @@ truncates or wraps — otherwise `BadTypeMismatch`.
|
||||
|
||||
At most 256 PLC/DATA bindings per control (excess are skipped and reported on `/hosts`).
|
||||
|
||||
## Alarm conditions (`ALARM:`, Gitea #831)
|
||||
|
||||
The control exposes one flat active-alarm list (`R_RI 27` first, `28` next — class, group, number, text; only refusal
|
||||
57 ends it). Besides the `Alarms/*` typed arrays (unchanged), the list can drive **native OPC UA Part 9 conditions**:
|
||||
a raw tag whose address is an `ALARM:` selector **and** whose `TagConfig` carries the cross-driver `alarm` object is
|
||||
exactly one condition, `ConditionId` = the tag's RawPath (the key the host routes and fans on — raw device folder +
|
||||
every referencing equipment folder, per the v3 native-alarm model). Same opt-in-per-tag shape as FOCAS #647 and
|
||||
MTConnect #626.
|
||||
|
||||
```jsonc
|
||||
{ "address": "ALARM:CLASS=7", "dataType": "Boolean",
|
||||
"alarm": { "alarmType": "AlarmCondition", "severity": 800 } }
|
||||
```
|
||||
|
||||
| Address | Condition is active while… |
|
||||
|---|---|
|
||||
| `ALARM:*` | any alarm is active on the control (summary) |
|
||||
| `ALARM:OTHER` | an alarm is active that no **filter** selector on the same control claims |
|
||||
| `ALARM:CLASS=<r>[/GROUP=<r>][/NUMBER=<r>]` (any subset, any order, each key once) | an alarm matches every named field |
|
||||
|
||||
`<r>` is `n` or `lo..hi` (inclusive; `..` because numbers can be negative — a real iTNC 530 reports `-1`). CLASS/GROUP
|
||||
are 0–65535 (wire uint16), NUMBER is int32. **No semantic class/group names**: their meaning is undocumented in the
|
||||
reference (pyLSV2 / the Java port keep them raw), so nothing is guessed — observe `Alarms/Classes` / `Alarms/Groups` or
|
||||
a condition message, then filter. One parser (`HeidenhainAlarmSelector`) backs the editor, the deploy inspector and the
|
||||
driver.
|
||||
|
||||
- **Opt-in is the tag — nothing is on by default.** An existing config gains no condition and loses nothing; the
|
||||
`Alarms/*` arrays keep publishing. This follows FOCAS #647 (`AlarmProjection.Enabled` became a no-op: the authored
|
||||
tag is the switch) and MTConnect #626 (opt-in because alarm tags leave the value subscription). There is no
|
||||
driver-level enable flag.
|
||||
- **Pairing is enforced** (`HeidenhainTagDefinitionFactory`, so the editor, the save path, CSV and the deploy inspector
|
||||
agree): an `ALARM:` address without the `alarm` object, or an `alarm` object on any other address, is refused. The
|
||||
tag's data type is fixed at `Boolean`; it is never a value (`ReadAsync` answers `BadNotSupported`), never in a data
|
||||
subscription, never writable (the `HeidenhainTagWritable` gate still applies). The typed editor seeds
|
||||
`{"alarmType":"AlarmCondition","severity":700}` when you pick an `ALARM:` selector and drops it when you leave one.
|
||||
- **Identity** = (class, group, number, **text**) — the number alone is not unique (`-1` above). **Displayed alarm** is
|
||||
deterministic (ascending class, group, number, text), never wire order. Message:
|
||||
`-1 Run test of cutout channels (class 7, group 4) [+N more]`.
|
||||
- **Transitions** (the FOCAS #647 rules): inactive → active `Raise`; a key not in the told set appears, or the displayed
|
||||
alarm leaves while others remain → `Retrigger` (un-acknowledges — a statement the operator has not seen); a
|
||||
non-displayed member leaves → nothing; nothing matches → `Clear`.
|
||||
- **Severity**: an authored integer `alarm.severity` is the exact Part 9 severity; otherwise High (700). No
|
||||
class→severity table is invented.
|
||||
- **Diffed against what the server was TOLD**, recorded only while a handler is attached; the record lives on the
|
||||
driver instance (survives a reconnect / same-set re-subscribe / in-place re-init). A deploy respawns the driver and its
|
||||
first walk re-states still-active alarms with `Raise`, which the host treats as a re-statement (ack kept,
|
||||
`NativeAlarmProjector` #647).
|
||||
- **A failed walk is not "no alarms."** A refusal (e.g. 23/49 access denied → `BadUserAccessDenied`, 22/24 →
|
||||
`BadNotSupported`), a list overflow (`MaxAlarms` → `BadOutOfRange`), a decode error, telemetry disabled, or comms loss
|
||||
(`BadNoCommunication`) keeps every condition's Active/Acked state and annotates its **quality** once (quality-only
|
||||
event, no `/alerts` row, one Warning per episode); the next good walk lifts it to Good before diffing. Only a down
|
||||
session degrades health (the existing rule) — a refusal never does.
|
||||
- **Acknowledgement is local-only.** LSV2 has no acknowledge request and the read-only allowlist is **not** widened:
|
||||
`AcknowledgeAsync` is a no-op success, so the Part 9 Acknowledge / AddComment the host routes to it is applied (and
|
||||
pair-replicated) server-side — the FOCAS / MTConnect posture. Confirm, the shelve methods and Enable/Disable are
|
||||
refused `BadNotSupported` by the host for every native condition. The control clears an alarm when its cause is reset;
|
||||
an operator ack never reaches it.
|
||||
- **Traffic:** the walk is the same allowlisted `R_RI 27/28` the arrays use, on the control's single acquisition loop
|
||||
every `Polling.AlarmMs`, and only for a control that carries an `ALARM:` tag. No new command type exists
|
||||
(`HeidenhainAlarmTests` asserts the wire log is a subset of the read set and `Lsv2ReadOnlyPolicy` is unchanged).
|
||||
- **Authoring:** the `/raw` tag modal (`Native alarm` section + the typed editor), manual entry (the editor seeds the
|
||||
alarm object) and CSV (`Alarm.AlarmType` / `Alarm.Severity` columns). The browse picker offers only catalog leaves.
|
||||
- E2E: `Runtime.Tests/Drivers/HeidenhainNativeAlarmEndToEndTests` (real driver → simulator → real `DriverHostActor`).
|
||||
|
||||
## Quality
|
||||
|
||||
| Condition | Value status / capability |
|
||||
@@ -183,6 +256,11 @@ overrides, axes, tool, the whole alarm walk) and aliased custom addresses are re
|
||||
- Subscriptions: the initial push and later publications are ordered under one lock, so a subscriber never
|
||||
receives an older value after a newer one. A re-init disposes every control loop first; a late start of a
|
||||
disposed loop is a no-op.
|
||||
- **Rediscovery signal (`IRediscoverable`, #831):** every session already reads `R_VR` at connect; when a control
|
||||
reconnects reporting a **different** identity (model, NC/PLC software version or option string) than the previous
|
||||
session on this driver instance, the driver raises `OnRediscoveryNeeded` once with both identities in the reason —
|
||||
the `/hosts` "re-browse" chip. Advisory only (drivers.md §Change Detection): nothing is re-materialised. The first
|
||||
session records the baseline; a deploy respawn forgets it (the next session is a new baseline). No extra traffic.
|
||||
- **Health:** `Degraded` only while a control's session is down (connect failing / lost), with
|
||||
`LastSuccessfulRead` frozen — the conjunction the #527 comms-loss watchdog needs. Per-tag refusals, denials and
|
||||
decode errors are quality, not health (else the watchdog would reconnect-loop a control that merely denies one
|
||||
@@ -202,7 +280,13 @@ session to the first device: INSPECT login + `R_VR`. A bare TCP listener does no
|
||||
any non-allowlisted telegram. The Java fixtures' captured iTNC 530 replies (`Fixtures/*.properties`) are replayed
|
||||
as golden vectors (inspect-only telemetry on two controls; DATA boundary replies on three).
|
||||
- `Runtime.Tests/Drivers/HeidenhainDriverEndToEndTests` — the real driver inside `DriverInstanceActor`.
|
||||
- `Configuration.Tests/DraftValidatorHeidenhainTests`, `AdminUI.Tests/Uns/HeidenhainAuthoringTests` + the fleet
|
||||
- `HeidenhainAlarmTests` (#831) — selector grammar, pairing, transition rules, and against the simulator: the captured
|
||||
iTNC 530 alarm record raising/clearing conditions, local-only ack (zero bytes sent), quality-only comms-loss/refusal,
|
||||
handler-detached recording, identity-change rediscovery; every case asserts the wire log stays inside the read set.
|
||||
- `Runtime.Tests/Drivers/HeidenhainNativeAlarmEndToEndTests` — driver → simulator → real `DriverHostActor` condition.
|
||||
- `Configuration.Tests/DraftValidatorHeidenhainTests`, `AdminUI.Tests/Uns/HeidenhainAuthoringTests`,
|
||||
`AdminUI.Tests/Uns/HeidenhainAuthoringUxTests` (#830 CSV preview ⇔ commit parity, data-type lock, single
|
||||
credential notice) + the fleet
|
||||
parity suites.
|
||||
- **LiveIntegration** (`--filter Category=LiveIntegration`, skipped unless `HEIDENHAIN_LSV2_HOST` is set; optional
|
||||
`HEIDENHAIN_LSV2_PORT`, `OTOPCUA_HEIDENHAIN_INSPECT_PASSWORD`): read-only INSPECT identity/selector/layout survey and a
|
||||
@@ -223,5 +307,5 @@ No real control was available for this port. Still open:
|
||||
|
||||
Writes of any kind, file upload/download, remote input/keyboard/motion, parameter edits, PLCDEBUG/MONITOR/FILE
|
||||
roles, scope setup, arbitrary requests, buffer-size negotiation, vendor SDK, serial transport, built-in SSH,
|
||||
PLC-wide symbol discovery / memory scanning, OEE mapping, and native alarms as Part 9 conditions (alarms are
|
||||
published as typed arrays).
|
||||
PLC-wide symbol discovery / memory scanning, OEE mapping, and any controller-side alarm acknowledgement (native alarm
|
||||
conditions are acknowledged server-side only — see [Alarm conditions](#alarm-conditions-alarm-gitea-831)).
|
||||
|
||||
@@ -81,6 +81,13 @@ for every field (security policy/mode, auth type, session timeout, keep-alive,
|
||||
browse root, node/depth caps). `ReconnectPeriod` was **deleted** (#676 — unused since #670 removed the SDK reconnect
|
||||
handler, and the AdminUI no longer shows it); stored configs that still carry `reconnectPeriod` keep parsing in the
|
||||
factory, the Test-Connect probe and the browse picker (unknown keys are skipped), and the form drops it on the next save.
|
||||
**Credentials (#815).** `Password` / `UserCertificatePassword` are a literal, `secret:opcuaclient/<name>` (the node's
|
||||
secret store) or `env:OTOPCUA_OPCUACLIENT_<NAME>`. They are resolved on the node and sent to the configured endpoint,
|
||||
so any other `secret:`/`env:` reference (and `file:`) is refused at save, deploy (`CredentialReferenceOutOfScope`), Test
|
||||
Connect, browse and (for the AdminUI browser) again before anything is read; an already-deployed driver's session open
|
||||
keeps the pre-#815 semantics in deprecated warn-only mode until its next deploy. Migrate an existing `secret:<name>` by re-creating
|
||||
the secret as `opcuaclient/<name>`. See [`docs/security.md`](../security.md) §"Credential references are namespaced per driver".
|
||||
|
||||
`ProbeTimeoutSeconds` (default 15, AdminUI "Probe timeout") caps the Test-Connect probe's deadline — it can shorten
|
||||
the budget the host hands the probe, never extend it (review 2026-10-21 O-4; it used to be authored and consumed by nothing).
|
||||
|
||||
@@ -108,7 +115,14 @@ The rule, for `opc.tcp` (and any scheme, as long as advertised and configured sc
|
||||
a real rewrite keeps the advertised path's percent-escaping (`AbsoluteUri`, #812);
|
||||
- a **different scheme** (an `https` endpoint next to an `opc.tcp` one) is a different transport and is left alone;
|
||||
- an unusable advertised URL (null / blank / unparseable) falls back to the configured one in the driver and the
|
||||
browser; the Client CLI/UI instead reports it (it validates before calling the rewriter).
|
||||
browser; the Client CLI/UI instead reports it (it validates before calling the rewriter);
|
||||
- an **IPv6 zone index** on the configured host (`opc.tcp://[fe80::1%eth0]:4840`) is **kept, verbatim** (#819).
|
||||
.NET's `Uri` parses the zone but drops it from `Host`/`Authority`/`AbsoluteUri`/`ToString()` — only
|
||||
`DnsSafeHost`/`IdnHost` carry it — so the rewriter splices it back into the string. The SDK dials
|
||||
`new Uri(EndpointUrl).IdnHost` (`TcpByteTransport` → `DnsEndPoint`), which keeps it, so a link-local endpoint now
|
||||
routes; before #819 the session dialled a zone-less `fe80::` address. The zone text is not normalised: write the
|
||||
form your OS resolves (`%eth0` / `%4`). .NET does **not** decode RFC 6874's `%25eth0` (its `DnsSafeHost` is
|
||||
`fe80::1%25eth0`), so that form is passed through as written — GetEndpoints and the session see the same string.
|
||||
|
||||
**What the rewrite does NOT touch.** It only chooses the network address. The **security policy and mode** are
|
||||
still selected from the server's advertisement exactly as before — the driver and browser filter on the configured
|
||||
@@ -118,7 +132,72 @@ user-token policies** are the advertised ones. **Certificate validation is uncha
|
||||
validates the certificate of the server actually reached, against the same trust store (the driver's PKI, the
|
||||
browse picker's separate `adminui-browse-pki`, the CLI's own) with the same `AutoAcceptCertificates` semantics. The
|
||||
NAT tests below pin that a secure browse through the NAT is refused on an untrusted certificate and succeeds once it
|
||||
is trusted.
|
||||
is trusted. The optional **host-name** check (#816, next section) is the one place the rewrite matters to validation:
|
||||
it runs against the rewritten URL, i.e. the **configured** host.
|
||||
|
||||
### Server certificate host name (#816)
|
||||
|
||||
**Finding (SDK 2.0.1).** The SDK only checks that the server certificate names the host you connected to when the
|
||||
session is created with `checkDomain: true`. All three surfaces — the driver (`OpenSessionOnEndpointAsync`), the
|
||||
AdminUI browser (`OpcUaClientDriverBrowser.OpenAsync`) and Client.Shared (`DefaultSessionFactory`) — called the
|
||||
8-argument `DefaultSessionFactory.CreateAsync(config, endpoint, updateBeforeConnect, sessionName, …)`, which
|
||||
forwards `checkDomain: false`. So **no host-name check ever ran**; trust was the trusted-peer / issuer store alone.
|
||||
That is why, in #813, a trusted NAT certificate for `otopcua-opc-plc-nat-private.invalid` browsed fine over an IP.
|
||||
|
||||
Where the check lives when enabled — **two** places in SDK 2.0.1, both calling
|
||||
`CertificateValidationExtensions.ValidateDomains` → `CertificateValidationCore.ValidateDomains` →
|
||||
`CertificateValidationHelpers.FindDomain`:
|
||||
|
||||
1. `DefaultSessionFactory.CreateChannelAsync` — before the channel opens, against the certificate in the
|
||||
**GetEndpoints reply** (`endpoint.Description.ServerCertificate`), whenever that reply carries one and **before any
|
||||
security-mode test**. opc-plc sends its certificate on its `None` endpoint too, so `checkDomain: true` refused a
|
||||
`None` connect (found in review, reproduced against `:50000`).
|
||||
2. `Session.OpenAsync` — after `CertificateManager.ValidateAsync` accepts the chain the server returned in
|
||||
CreateSession (only when the channel or the user-token policy needs the server certificate).
|
||||
|
||||
`FindDomain` compares `ConfiguredEndpoint.EndpointUrl.IdnHost` with the names `X509Utils.GetDomainsFromCertificate`
|
||||
returns (DNS / IP SANs, and the subject's `DC=` parts); IP literals are normalised, and a `localhost` / `127.0.0.1` /
|
||||
`::1` URL is compared as the machine's own host name + FQDN. A miss is `BadCertificateHostNameInvalid` ("The domain
|
||||
'…' is not listed in the server certificate."), passed to the certificate manager's `AcceptError` hook first.
|
||||
Consequences:
|
||||
|
||||
- **It checks the CONFIGURED host.** `ConfiguredEndpoint.EndpointUrl` is the URL `OpcUaEndpointSelection` /
|
||||
`EndpointSelector` rewrote, so the advertised host the server named itself never takes part.
|
||||
- **Sign / SignAndEncrypt endpoints only — enforced here, not by the SDK.** Because of site 1, all three call sites
|
||||
pass `checkDomain` only when the selected endpoint's `SecurityMode != None` (`OpcUaEndpointSelection.CheckDomainFor`
|
||||
for the driver and the browser, the same expression in Client.Shared's `ConnectToEndpointAsync`; both cover the
|
||||
failover sweep, which goes through the same per-endpoint open). A host-name check on an unauthenticated channel
|
||||
proves nothing.
|
||||
- **It is a loose match.** For a DNS host the SDK also accepts a match on the **first label alone**:
|
||||
`plc1.site-a.corp` passes a certificate whose only SAN is `PLC1` (case-insensitive), and so would
|
||||
`plc1.site-b.corp`. There is no wildcard support (`*.site-a.corp` matches nothing), and the subject's `DC=`
|
||||
components count as names.
|
||||
- **A zone-qualified IPv6 host never passes.** `FindDomain` compares the host *with* its zone (`fe80::1%eth0`; macOS
|
||||
normalises `%en0` to `%14`) against the certificate's zone-less IP SANs, so a link-local endpoint (#819) needs
|
||||
`validateServerHostName: false`. The repo does not fork the SDK check to strip the zone.
|
||||
- **`autoAcceptCertificates` accepts a mismatch too** (the driver's auto-accept `AcceptError` handler returns `true`
|
||||
for everything but revocation). The Client CLI always auto-accepts.
|
||||
- Test Connect (`OpcUaClientDriverProbe`) only calls GetEndpoints on an unsecured discovery channel — no session, so no
|
||||
certificate or host-name check.
|
||||
|
||||
**Policy: `certificateValidation.validateServerHostName`** (driver JSON; AdminUI driver form "Server host-name check";
|
||||
the browser reads the same JSON):
|
||||
|
||||
| Value | Meaning |
|
||||
|---|---|
|
||||
| absent / `null` (**default**) | `OpcUaCertificateValidationOptions.DefaultValidateServerHostName` = **off** — exactly the pre-#816 behaviour, so no existing deployment changes on upgrade. The AdminUI form keeps it absent ("Default (off)") rather than writing a concrete `false`. |
|
||||
| `true` | Validate: the certificate must list the configured endpoint host. The driver's connect failure then appends the remedy (fix the certificate, connect by a name it lists, or opt out). |
|
||||
| `false` | Explicit opt-out — the NAT / container port-publish / connect-by-an-IP-the-cert-omits case. |
|
||||
|
||||
When to turn it on: it earns its keep when trust comes from a **CA** (a trusted issuer vouches for every certificate
|
||||
it signed, so only the host check ties the certificate to *this* server) — within the limits above: a
|
||||
first-label-only match means a certificate issued as `PLC1` for one site is accepted for `plc1` at every site, so give
|
||||
certificates fully-qualified, site-unique names. With the server's own self-signed
|
||||
certificate pinned in the trusted-peer store it adds little — the pin already identifies the server. Client
|
||||
CLI: `--validate-host-name` (`ConnectionSettings.ValidateServerHostName`, default `false`). Pinned by
|
||||
`ServerHostNameValidationTests` (driver vs. the plain opc-plc, whose certificate names only its container id) and
|
||||
`NatPublishedEndpointBrowseTests` (browser vs. the NAT opc-plc): unset/`false` connect, `true` is refused naming the
|
||||
configured host.
|
||||
|
||||
**Running the NAT tests.** `NatPublishedEndpointBrowseTests`
|
||||
(`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests`, `Fixture=opc-plc-nat`) run against
|
||||
@@ -133,13 +212,20 @@ OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test \
|
||||
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat
|
||||
```
|
||||
|
||||
The secure case asserts a certificate **refusal** by default. To make it positive, trust the fixture's server
|
||||
certificate in the browse PKI and set `OPCUA_NAT_SERVER_CERT_TRUSTED=1` — the certificate is regenerated whenever
|
||||
the container is recreated, so fetch the current one:
|
||||
`ssh dohertj2@10.100.0.35 'docker exec otopcua-opc-plc-nat sh -c "cat /app/pki/own/certs/*.der"' > "<LocalAppData>/OtOpcUa/adminui-browse-pki/trusted/certs/opc-plc-nat.der"`
|
||||
(`~/Library/Application Support/...` on macOS, `~/.local/share/...` on Linux), and delete it again afterwards. CI
|
||||
(`v2-integration.yml`) starts its own throwaway NAT copy (`scripts/ci/integration-fixtures.sh`); `v2-e2e.yml` points at
|
||||
the shared one. Neither trusts the certificate, so CI runs the refusal form.
|
||||
Every secure browse there runs on a **throwaway browse PKI** (an internal PKI-root seam on the browser), never your
|
||||
`adminui-browse-pki`. The refusal case trusts nothing and always runs. The positive case (unset / `false` host check)
|
||||
and the host-name refusal (`true`) trust the fixture's **public** certificate (#818), from:
|
||||
|
||||
1. `OPCUA_NAT_SERVER_CERT` — a DER or PEM file; must be the certificate the server presents, or the test **fails**; else
|
||||
2. the committed `tests/.../Driver.OpcUaClient.IntegrationTests/Docker/nat/opc-plc-nat-server.der` — the shared-host
|
||||
stack's certificate, stable because its PKI lives on the `opc-plc-nat-pki` volume. If it is not the certificate
|
||||
the server presents (another NAT fixture, or the stack's certificate was rolled) these tests **skip**.
|
||||
|
||||
The private key never leaves the volume. The committed certificate expires **2027-11-10**; to roll it, see the comment
|
||||
at the top of `Docker/nat/docker-compose.yml`. CI (`v2-integration.yml`) starts its own throwaway NAT copy
|
||||
(`scripts/ci/integration-fixtures.sh`), copies that container's public certificate out with `docker exec` and exports
|
||||
`OPCUA_NAT_SERVER_CERT`, so it runs the positive cases too; `v2-e2e.yml` points at the shared stack and uses the
|
||||
committed certificate.
|
||||
|
||||
### Session lifetime
|
||||
|
||||
@@ -379,6 +465,7 @@ also refuses them explicitly — defence in depth (#277), not a behaviour change
|
||||
|---|---|---|
|
||||
| `rejectUnknownRevocationStatus` | `false` | Reject a server certificate whose revocation status cannot be determined (no CRL for its issuer, or a stale one). Leave `false` for brownfield sites without CRL infrastructure. **Only effective with `autoAcceptCertificates` off** — the auto-accept handler accepts every suppressible error, including revocation-unknown. |
|
||||
| `issuerStorePath` | `null` (= `%LocalAppData%/OtOpcUa/pki/issuers`) | Directory used as the trusted-issuer store. The SDK reads issuer CRLs (`*.crl`) from its `crl/` sub-folder, so this is how an operator points the driver at a managed CRL set. |
|
||||
| `validateServerHostName` | `null` (= off) | #816: require the server certificate to name the **configured** endpoint host (`BadCertificateHostNameInvalid` otherwise). Sign / SignAndEncrypt endpoints only (never passed for `None`); accepted anyway under `autoAcceptCertificates`; loose (first-label) match; never matches a zone-qualified IPv6 host. `false` = explicit NAT / link-local opt-out. See "Server certificate host name" above. |
|
||||
|
||||
### Subscription tuning (#273)
|
||||
|
||||
|
||||
@@ -280,8 +280,9 @@ Razor picker body BrowserSessionService IDriverBrowser
|
||||
(`ssh dohertj2@10.100.0.35 'cd /opt/otopcua-opcuaclient-nat && docker compose up -d'`, compose in
|
||||
`tests/.../Driver.OpcUaClient.IntegrationTests/Docker/nat/`), and CI starts a throwaway copy. Run:
|
||||
`OPCUA_NAT_ENDPOINT=opc.tcp://10.100.0.35:50010 dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser.IntegrationTests --filter Fixture=opc-plc-nat`.
|
||||
The secure case asserts a certificate refusal unless the fixture cert is trusted in the browse PKI and
|
||||
`OPCUA_NAT_SERVER_CERT_TRUSTED=1` (see `docs/drivers/OpcUaClient.md`).
|
||||
The secure cases run on a throwaway browse PKI: a refusal with nothing trusted, and (#818) positive / #816
|
||||
host-name cases that trust the fixture's public certificate (`OPCUA_NAT_SERVER_CERT`, or the committed
|
||||
`Docker/nat/opc-plc-nat-server.der`) — see `docs/drivers/OpcUaClient.md`. (`OPCUA_NAT_SERVER_CERT_TRUSTED` is retired.)
|
||||
- No Galaxy integration suite in v1 (requires wonder-app-vd03; deferred)
|
||||
|
||||
### Specific regression tests
|
||||
|
||||
+123
-2
@@ -647,9 +647,13 @@ and Write under old and new names until the address space re-points them. An id
|
||||
folder / virtual tag / condition, a probe, a NodeDeleted model change) is decided only by a `Cluster` or `uns`
|
||||
`Namespace` grant — so narrow grantees do not receive NodeDeleted for removed opaque-id nodes, and a removed id can
|
||||
never match an unrelated narrow grant that shares its name; a four-segment (reference-shaped) id keeps its split.
|
||||
Identifiers containing NUL are refused. Server-object (`i=2253`) delivery of a
|
||||
Identifiers containing NUL are refused. A UNS variable the reference pass wired to its raw node (the `Organizes`
|
||||
edge) is placed as a reference variable whatever its id spells (Gitea #826 — the id rule alone made an equipment's
|
||||
references folder-owned when an Area was named exactly like that equipment's id, losing their `Tag`-grant match).
|
||||
Server-object (`i=2253`) delivery of a
|
||||
**scripted** condition is now decided on the condition alone (its folders add nothing in steady state and would
|
||||
leak a moved alarm to its old owner). So a key holding only an `Equipment` grant can navigate to its equipment,
|
||||
leak a moved alarm to its old owner). Direct event items on a folder get the same per-event decision (Gitea #826,
|
||||
below). So a key holding only an `Equipment` grant can navigate to its equipment,
|
||||
read its alarm children, receive its scripted alarm events through the Server object and Acknowledge / Confirm /
|
||||
AddComment with the EventId it received. Fail-closed throughout: an unplaceable node is denied under every grant.
|
||||
| HistoryRead (all four variants) | **nothing** | `HistoryRead` on each node's scope, decided **per node in the batch** |
|
||||
@@ -756,6 +760,31 @@ Behaviours worth knowing before enabling it:
|
||||
condition) runs on removal, rebuild and Server-object subscribe/delete. A deploy of N alarms under a
|
||||
live Server-object item is therefore N bookkeeping steps, not N²/2 under the node-manager lock.
|
||||
|
||||
**Direct (per-folder) event items are filtered per event too** (Gitea #826). An item on an equipment or raw
|
||||
device folder is fed by the SDK's own per-notifier fan-out (`MonitoredNode2`), which the #586 dispatcher never
|
||||
touches — so after a deployment moved a scripted alarm (or dropped the reference projecting a native alarm into
|
||||
an equipment), the OLD folder's subscribers kept receiving it, EventId included, until the rebuild landed, and
|
||||
indefinitely after a failed apply (commands were already denied). With the flag **on**, each event to such an
|
||||
item now also needs, against the CURRENT state:
|
||||
- the event's condition to be the **live** instance (a torn-down one is withheld);
|
||||
- the folder to still be a legitimate notifier of it — a node it hangs beneath, or a folder it is wired into
|
||||
**that the current generation still projects its raw tag into** (`DataPlaneAuthoritySnapshot.TagReferencingEquipment`,
|
||||
`RawPath → EquipmentIds`, decoded from the artifact's `UnsTagReferences` by the same resolver as the grants);
|
||||
- and the condition to pass exactly the Server-object rule above for the item's session.
|
||||
|
||||
The generation check also narrows the **Server-object** rule: a wired folder whose reference the generation
|
||||
dropped no longer admits the native alarm there either. **`ConditionRefresh`** on a folder item is filtered the
|
||||
same way. The hook is `INodeManager3.ValidateEventRolePermissions`, which the node manager re-implements (the
|
||||
base method is not virtual); the SDK memoizes that verdict per `(item, event type, SourceNode)`, so the node
|
||||
manager clears each notifier's cache whole (sessions and orphaned, session-less subscriptions alike — an orphaned
|
||||
subscription must not queue a moved alarm's events for a later `TransferSubscriptions`) on every new generation or
|
||||
authorizer, on each `AclRevalidationTimer` tick, and at the end of every apply, and not at all while enforcement is
|
||||
off. A verdict is decided on the condition currently live under the event's NodeId, so a replaced instance's late
|
||||
event cannot leave a deny behind for its successor. Only alarm-bearing raw tags are carried in the reference map. Withheld events are not replayed and the client is not told; restoring the reference or
|
||||
grant resumes the feed on the next event. ⚠️ A native alarm reaches an equipment folder's items under
|
||||
enforcement only while the installed generation carries that reference — a hand-built authority (tests) must
|
||||
supply `tagReferencingEquipment`. Inert with the flag off.
|
||||
|
||||
**Model-change and audit events on the Server object are gated too** (Gitea #588). The node manager's
|
||||
own server-wide events — `GeneralModelChangeEvent` (NodeAdded / NodeDeleted / DataTypeChanged) and the
|
||||
`AuditWriteUpdateEvent` of a rejected device write — used to go straight to `Server.ReportEvent`, where
|
||||
@@ -1191,6 +1220,98 @@ Supporting hygiene, none of which is the control:
|
||||
If a deployment needs the browse surface narrower than "anyone who may author config", the lever is the
|
||||
page policy (and the LDAP group→role mapping behind it), not the browser.
|
||||
|
||||
#### Credential references are namespaced per driver (#815)
|
||||
|
||||
The outbound-connect capability above has a sharper edge whenever the config carries a **credential
|
||||
reference**: a Designer authors both the endpoint *and* the reference, and the node resolves the reference
|
||||
and sends the value to that endpoint — on Test Connect, on a live browse, and at runtime. Before #815 an
|
||||
unscoped reference let anyone who may author config exfiltrate any value the admin node could read, with no
|
||||
deploy: point a Galaxy `Gateway.Endpoint` at their own host, set `ApiKeySecretRef` to
|
||||
`env:ConnectionStrings__ConfigDb` (or `env:ServerHistorian__ApiKey`, or `file:<any path>`), and click Test
|
||||
Connect / browse; or set an OpcUaClient `Password` to `secret:<any store entry>`.
|
||||
|
||||
Every operator-authored reference is now confined to its driver's namespace — one table,
|
||||
`DriverCredentialPolicy` (`Core.Abstractions`), one parser/resolver, `CredentialReferences`:
|
||||
|
||||
| Driver | Fields | Allowed references | Literal |
|
||||
|---|---|---|---|
|
||||
| `GalaxyMxGateway` | `Gateway.ApiKeySecretRef` | `secret:galaxy/<name>`, `secret:otopcua/galaxy/<name>`, `env:OTOPCUA_GALAXY_<NAME>`, legacy `env:GALAXY_MXGW_<NAME>` | allowed (warns), `dev:<key>` |
|
||||
| `GalaxyMxGateway` | `AlarmAck.FallbackPasswordSecretRef` | same | `dev:<value>` only |
|
||||
| `OpcUaClient` | `Password`, `UserCertificatePassword` | `secret:opcuaclient/<name>`, `env:OTOPCUA_OPCUACLIENT_<NAME>` | allowed |
|
||||
| `Heidenhain` | `InspectPassword`, `DncPassword`, `DataPassword` | `secret:heidenhain/<name>`, `env:OTOPCUA_HEIDENHAIN_<NAME>` (#803) | refused |
|
||||
|
||||
Rules (all drivers): the prefixes are lower-case and exact (`SECRET:` / `Env:` are refused, never treated
|
||||
as a literal); `${secret:<name>}` is accepted as a spelling of `secret:<name>`; secret names are
|
||||
`[A-Za-z0-9._-]` segments with no empty, `.` or `..` segment; the variable suffix is `[A-Z0-9_]+`; **`file:` is
|
||||
refused everywhere** (Galaxy used to read any file the node could open). A field is matched at any depth of
|
||||
the DriverConfig / DeviceConfig, case-insensitively, as the drivers bind it.
|
||||
|
||||
Enforced, fail-closed, at every seam — the earlier ones exist for a clear message, the last one is the guarantee:
|
||||
|
||||
1. **AdminUI save** — `DriverConfigSaveGate` (`RawTreeService` driver/device create + update): the config is
|
||||
never stored.
|
||||
2. **Deploy gate** — `CredentialReferenceDeployChecks`, code **`CredentialReferenceOutOfScope`** (Heidenhain keeps
|
||||
`HeidenhainCredentialPersisted`).
|
||||
3. **Test Connect** — `AdminOperationsActor` refuses before any probe runs (`"Test Connect refused: …"`).
|
||||
4. **Live browse** — `BrowserSessionService.OpenAsync` refuses before any bespoke or universal browser opens.
|
||||
5. **Resolve time** — `GalaxySecretRef`, `GalaxyAckCredentialResolver`, `OpcUaUserIdentityFactory` and
|
||||
`HeidenhainCredentials` all resolve through `CredentialReferences`, which re-checks the scope **before** any
|
||||
secret-store or environment read. This is hard-fail on every admin-node path (the Galaxy and OpcUaClient
|
||||
AdminUI browsers, every Test Connect probe) and everywhere for Heidenhain. The one exception is the deprecated
|
||||
warn-only compatibility path for already-deployed drivers, below.
|
||||
|
||||
Messages name the field and the allowed forms, never the value. Out of scope by design: `${secret:}` tokens in
|
||||
`appsettings.json` (host config, not operator-authored), and the `Sql` driver's `connectionStringRef`, which is
|
||||
already a fixed prefix (`Sql__ConnectionStrings__<ref>`) whose value carries its own server — a Designer can
|
||||
choose *which* provisioned database a ref names, but cannot redirect its credentials to another host.
|
||||
|
||||
**Migration (existing configs).** A stored config with an out-of-namespace reference is listed by the upgrade
|
||||
preflight (below), refused by its next deploy (`CredentialReferenceOutOfScope` — the message names the
|
||||
migration), and refused by Test Connect / live browse. Values most likely to be in real config databases, with
|
||||
the exact replacement (create the new secret / variable first — on every node that runs the driver **and** on
|
||||
the admin nodes, which run Test Connect and browse — then edit the field):
|
||||
|
||||
| Old value | Why it is refused | Replace with |
|
||||
|---|---|---|
|
||||
| Galaxy `env:MX_API_KEY` — the **old AdminUI Galaxy form default** (pre-#815 `GalaxyDriverForm.razor`), so the likeliest value in a real DB | variable outside the namespace | rename the variable to `OTOPCUA_GALAXY_API_KEY` and set `env:OTOPCUA_GALAXY_API_KEY` (or store it and use `secret:galaxy/<inst>/apikey`) |
|
||||
| Galaxy `env:<ANY_OTHER_NAME>` (e.g. `env:MY_API_KEY`) | same | variable `OTOPCUA_GALAXY_<NAME>` → `env:OTOPCUA_GALAXY_<NAME>`. `env:GALAXY_MXGW_API_KEY` (docker-dev, parity docs) needs **no** change |
|
||||
| Galaxy `file:<path>` — `docs/Configuration.md` used to recommend it for production | `file:` is refused everywhere | put the file's contents in the store as `galaxy/<inst>/apikey` → `secret:galaxy/<inst>/apikey` (or a variable `OTOPCUA_GALAXY_<NAME>`) |
|
||||
| Galaxy `secret:<name>` not under `galaxy/` (e.g. the old `docs/v2/driver-specs.md` example `secret:galaxy-gw-api-key`) | secret outside the namespace | re-create it as `galaxy/<name>` (e.g. `galaxy/gw/apikey`) → `secret:galaxy/gw/apikey`. Names already under `galaxy/` or `otopcua/galaxy/` need no change |
|
||||
| Galaxy `Env:X`, `SECRET:x`, `File:x`, … (any non-lower-case prefix) — **pre-#815 resolved these case-insensitively** | prefixes are now exact | the lower-case namespaced form from the rows above |
|
||||
| Galaxy `AlarmAck.FallbackPasswordSecretRef` = `env:` / `file:` / `secret:` outside the namespace | same | as above; the documented `secret:otopcua/galaxy/<name>` needs no change |
|
||||
| OpcUaClient `Password` / `UserCertificatePassword` = `secret:<name>` not under `opcuaclient/` (e.g. `secret:opcua/plant/pw`) | secret outside the namespace | re-create the secret as `opcuaclient/<name>` → `secret:opcuaclient/<name>` |
|
||||
| OpcUaClient `SECRET:x` / `Secret:x` — pre-#815 resolved case-insensitively | prefixes are now exact | `secret:opcuaclient/<name>` |
|
||||
| OpcUaClient **literal** password that happens to start with `env:`, `file:`, `secret:`/`${secret:` in any casing (e.g. a real password `Env:Prod1`) — pre-#815 sent it verbatim | it now parses as a reference | store it: `secret:opcuaclient/<name>` (a literal that does not start with one of those prefixes is unaffected) |
|
||||
|
||||
**Warn-only compatibility for already-deployed drivers (⚠️ deprecated — a later release fails closed).** Refusing
|
||||
at resolve time would fault running plant drivers at the upgrade restart — and, for Galaxy, at **every gateway
|
||||
reconnect**: `GalaxyDriver.ReopenAsync` (the `ReconnectSupervisor` callback) rebuilds the client options through
|
||||
`BuildClientOptionsAsync`, which re-resolves `Gateway.ApiKeySecretRef` each time, so a running driver would fault
|
||||
at its next reconnect, not at a deploy. So, for one compatibility window, the running-driver call sites — Galaxy
|
||||
`InitializeAsync` / `ReopenAsync` / the ack-fallback resolver, and OpcUaClient session open — resolve an
|
||||
out-of-namespace reference with the **old** semantics (Galaxy: case-insensitive `env:`/`file:`/`dev:`/`secret:`,
|
||||
literal otherwise; OpcUaClient: case-insensitive `secret:`, literal otherwise) and log a Warning naming the
|
||||
driver, field and allowed form (never the value), at most once per field per hour. The use also rides the
|
||||
driver's diagnostics bag (`credential-compat:<field>`) to `/hosts` and to the `/raw` credential banner. This is
|
||||
chosen **by call site**, not by a flag (`LegacyCredentialReferenceTracker`): the AdminUI browsers and every probe
|
||||
never get the tracker and stay hard-fail, and Heidenhain (new driver, no legacy configs) is hard-fail everywhere.
|
||||
It cannot be used to exfiltrate: save, Test Connect, browse and the deploy gate all refuse such a reference, so it
|
||||
can only be the reference an administrator deployed **before** the upgrade, and it lives only until that driver's
|
||||
next successful deploy (which requires migrating it). The hard-fail release deletes the tracker, the per-driver
|
||||
`Legacy…` resolve methods and their `legacy` parameters — nothing else changes.
|
||||
|
||||
**Upgrade preflight.** On an admin node, `CredentialReferencePreflightService` runs once, 15 s after
|
||||
`ApplicationStarted` (never inside an Akka starter, and it resolves nothing — ConfigDb reads only). It scans
|
||||
every credential-resolving driver's stored `DriverConfig` / `DeviceConfig` rows (**Draft**) and the latest
|
||||
deployment artifact (**Deployed** — what the running drivers were given) and logs one Warning per finding:
|
||||
driver, device, field, allowed form, never the value. The same list, plus running drivers that reported a
|
||||
warn-only resolution (**Running**, from driver-health diagnostics, fleet-wide), is shown as a banner on `/raw`,
|
||||
with a Re-check button.
|
||||
|
||||
Residual (accepted): within one namespace, a Designer can still point one driver instance at another
|
||||
instance's secret (e.g. `secret:galaxy/<other-inst>/apikey`) — the value then reaches the endpoint the Designer
|
||||
chose. Per-instance scoping is a possible follow-up.
|
||||
|
||||
### Role grant source
|
||||
|
||||
Admin reads `LdapGroupRoleMapping` rows from the Config DB (`src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/LdapGroupRoleMapping.cs`) — the same pattern as the data-plane `NodeAcl` but scoped to Admin roles + (optionally) one cluster for multi-site fleets (a system-wide row, `IsSystemWide = true`, stacks additively with cluster-scoped rows). The `RoleGrants.razor` page lets `Administrator`s edit these mappings without leaving the UI.
|
||||
|
||||
@@ -51,7 +51,7 @@ Per `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Config/GalaxyDriverOptions.cs`:
|
||||
{
|
||||
"Gateway": {
|
||||
"Endpoint": "http://localhost:5120",
|
||||
"ApiKeySecretRef": "secret:galaxy-gw-api-key",
|
||||
"ApiKeySecretRef": "secret:galaxy/gw/apikey",
|
||||
"UseTls": true,
|
||||
"CaCertificatePath": null,
|
||||
"ConnectTimeoutSeconds": 10,
|
||||
|
||||
Reference in New Issue
Block a user