dohertj2/lmxopcua
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: within-timestamp tie-cluster paging + AbCip/TwinCAT UDT member discovery

Browse Source
This commit is contained in:
Joseph Doherty
2026-06-17 20:35:34 -04:00
parent 4a7b0fde7b
commit 163f57b6bc
3 changed files with 106 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Historian.md
+27 -11
View File
@@ -64,7 +64,8 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
"Port": 32569,
"UseTls": false,
"ServerCertThumbprint": "",
"SharedSecret": ""
"SharedSecret": "",
"MaxTieClusterOverfetch": 65536
}
}
```
@@ -77,6 +78,7 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
| `UseTls` | bool | `false` | Wrap the TCP connection in TLS. |
| `ServerCertThumbprint` | string | — | Optional SHA-1 thumbprint to pin the sidecar's TLS certificate. Leave empty for CA-chain validation. |
| `SharedSecret` | string | — | Shared secret token the sidecar expects on every connection. Required when `Enabled`. |
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
> **Do not commit `SharedSecret` to `appsettings.json`.** Set it via an environment variable,
> a secrets store, or a deployment-time overlay. The checked-in default is always empty.
@@ -141,16 +143,30 @@ paging time-based:
SourceTimestamp *inclusive* and drops the boundary samples already emitted, so samples sharing
the boundary timestamp are neither duplicated nor skipped.
> **Paging limitation — oversized tie clusters.** The tie-safe cursor is a `(timestamp, skip)`
> pair, and the single-shot backend only accepts `(start, end, cap)` — it cannot skip. So if **more
> samples share one `SourceTimestamp` than `NumValuesPerNode`** (a tie cluster larger than the page
> cap), the cursor cannot advance past that timestamp: every resume re-reads the same first `cap`
> ties. Rather than silently truncate the read to `GoodNoData` (which would permanently drop the
> un-emitted ties), the resume read fails that node **loudly** with
> `BadHistoryOperationUnsupported` and logs the tag + timestamp + cap. The operator's remedy is to
> re-issue the read with a larger `NumValuesPerNode`. For a single tag's raw history this is a data
> anomaly (raw samples normally carry strictly increasing distinct timestamps); a fully cursor-based
> fix that pages *within* a single timestamp is a possible follow-up.
> **Oversized tie clusters — within-timestamp paging.** When more samples share one
> `SourceTimestamp` than the current page cap, the server detects that the cursor has stalled on
> a tie cluster (the last returned timestamp equals the resume timestamp). It then **over-fetches
> the entire cluster** at that single timestamp up to a bounded ceiling controlled by
> `ServerHistorian:MaxTieClusterOverfetch` (default **65 536**), then serves the cluster
> `NumValuesPerNode` samples at a time across successive pages, advancing the cursor one tick past
> the timestamp once the cluster is fully drained.
>
> **Short pages within a cluster still carry a continuation point.** A within-cluster page that
> returns fewer than `NumValuesPerNode` samples (because the cluster happened to be smaller than
> the cap, or is the final partial batch) is not the last page if the cluster itself has not been
> fully emitted — the server retains the continuation point so the client can drain the remainder.
> Only when the cluster is exhausted and the cursor has advanced past the timestamp does the
> short-page rule apply.
>
> **Cluster larger than `MaxTieClusterOverfetch`.** If the over-fetch itself reaches the ceiling
> without spanning the full cluster, the node fails **loudly** with
> `BadHistoryOperationUnsupported` and the tag + timestamp + ceiling are logged. Remedies: raise
> `MaxTieClusterOverfetch` (or `NumValuesPerNode`) to cover the full cluster, or investigate the
> data anomaly (raw samples normally carry strictly increasing distinct timestamps).
>
> For a single tag's raw history a tie cluster larger than the default 65 536 is a severe data
> anomaly. The ceiling exists to bound server-side memory on pathological data, not to cap normal
> operation.
Continuation points are bound to the OPC UA session (the SDK's
`ServerConfiguration.MaxHistoryContinuationPoints` cap, default 100, with oldest-eviction; points
docs/drivers/AbCip.md
+37
View File
@@ -119,6 +119,43 @@ integration fixture is down during normal dev (Docker host fixture currently off
See [Uns.md §Array tags](../Uns.md#array-tags-1-d) for the cross-driver coverage matrix
and the UI authoring flow.
## Controller-discovered UDT member variables
When `EnableControllerBrowse` is set, the driver walks the controller's `@tags` symbol table and
surfaces each UDT-typed tag's **atomic member leaves** as individually addressable OPC UA
Variables — in addition to the UDT container tag itself.
### What is discovered
- **Top-level UDT members** — the Template Object for each controller-browse UDT is fetched via
its template instance ID; each atomic member (BOOL, SINT, INT, DINT, REAL, …) becomes a
separate Variable node under the UDT's `Discovered/` sub-folder, addressable by its member
path (e.g. `MyUdt.Temperature`, `MyUdt.Flags`). Top-level atomic member discovery is
**functional in production**.
- **Nested struct members** — when a UDT member is itself a struct, the driver walks into it up
to a **depth cap of 8 levels**. Nested-struct expansion is a **documented deferral in
production**: the Template Object member block carries no nested template ID, so the
sub-shape cannot be re-fetched from the controller at discovery time. Nested struct leaves are
never mis-emitted — they are simply dropped (the parent member is omitted from discovery if it
is not atomic and its sub-shape is unavailable). Use pre-declared `Members` for nested structs
that must be individually addressed.
### Bare-container reads
Reading the UDT container tag itself (without a member suffix) returns
`BadNotSupported` — address the atomic member leaves instead.
### Member writes
UDT member **writes** discovered via controller browse are **deferred** in this release.
Pre-declared `Members` with `Writable: true` retain full read/write support as before.
### Configuration
No new configuration keys are required. Set `EnableControllerBrowse: true` in the
`AbCipDriverOptions` JSON to enable the `@tags` walk; UDT member expansion is automatic for all
UDT-typed tags found in the walk.
## Operational Notes
- **Native heap is invisible to the GC.** `GetMemoryFootprint()` reports CLR allocations only; libplctag's native `Tag` heap does not show up there. Watch whole-process RSS, and use `ReinitializeAsync` (tears down + re-creates every device's libplctag handles) as the remediation for native-heap growth.
docs/drivers/TwinCAT.md
+42
View File
@@ -154,6 +154,48 @@ on PLC re-download (unchanged semantics from scalar nodes).
See [Uns.md §Array tags](../Uns.md#array-tags-1-d) for the cross-driver coverage matrix
and the UI authoring flow.
## Controller-discovered struct/UDT/FB member variables
When `EnableControllerBrowse` is set, `BrowseSymbolsAsync` walks the device's symbol tree
recursively and expands **Struct / UDT / Function-Block typed symbols** into their **atomic
member leaves** as individually addressable OPC UA Variables, surfaced under the `Discovered/`
sub-folder.
### What is discovered
- Each symbol of a struct, UDT, or FB type is recursively expanded. The walk follows the ADS
symbol tree's `SubSymbols` collection.
- **Full instance paths** are preserved for every leaf (e.g. `MAIN.Motor.Speed`,
`GVL.Drive1.Status.Running`).
- The recursion is **depth-capped at 8 levels** to guard against pathological or circular-like
layouts.
- **Unsupported leaves** (types the driver cannot map to an OPC UA data type) are silently
dropped rather than surfaced as error nodes.
### Pre-declared Structure-typed tags
Pre-declared `Tags` with a `DataType` of `Structure` are **still rejected** — the driver cannot
read an opaque struct blob without knowing its member layout. For structs that must be
individually addressed, use `EnableControllerBrowse` to let the driver discover the member
layout from the controller's symbol table, rather than pre-declaring the container tag.
### Member writes
Discovered struct/UDT/FB member **writes** are **deferred** in this release. Scalar atomic
members can be written when pre-declared with `Writable: true`; the controller-browse expansion
path does not yet wire the write channel.
### Live caveat — Flat-mode vs VirtualTree-mode browse
The ADS client's `BrowseSymbolsAsync` may return symbols in **Flat mode** (a flat list of all
instance paths, with no `SubSymbols` populated) on some TC3 runtime configurations, in which
case the recursive struct expansion produces no sub-symbols and the walk yields only top-level
symbols. If a live TC3 browse does not populate `SubSymbols` on struct-typed symbols, the
alternative is **VirtualTree mode** (which returns a hierarchical symbol tree with `SubSymbols`
populated). VirtualTree-mode browse is an unverified follow-up; if Flat-mode struct expansion
produces empty `Discovered/` sub-folders for struct symbols, switch to VirtualTree mode via the
driver configuration.
## Further reading
- [`docs/v2/driver-specs.md §6`](../v2/driver-specs.md) — full per-field spec and