Commit Graph

201 Commits

Author SHA1 Message Date
Joseph Doherty 945ec2c069 Merge feat/677-aggregates: reverse processed reads SDK-anchored, start-of-data Partial (forward), AggregateConfiguration accept-defaults, IntegralDivisor=1, SDK parity tests (#677) 2026-10-11 11:47:52 -04:00
Joseph Doherty 029a916288 Merge feat/674-historyupdate-posture follow-ups: HistoryUpdate closed until first redundancy snapshot, node-level v1 refusal, host wiring tests, plan-before-gate, UTC skew (#674 review) 2026-10-11 11:41:36 -04:00
Joseph Doherty 232c03ed1b fix(historian): #674 review follow-ups — HistoryUpdate boot-window gate, node-level v1 refusal, plan-before-gate, UTC skew compare
- HistoryUpdate Primary gate now reads IHistoryUpdateRoleView (PrimaryGatePolicy.ShouldServeHistoryUpdate):
  the data-plane verdict but CLOSED until this node's first redundancy snapshot, so both pair nodes cannot
  accept in the pre-join boot window. Single nodes open on their own singleton's first snapshot; admin-only
  nodes register no view => no gate. DriverHostActor publishes it at every Primary-verdict publish point.
- SqlServer v1: the updater throws NotSupportedException on every arm; the node manager maps it to a
  node-level BadHistoryOperationUnsupported (was per-item under a Good node for UpdateData/DeleteAtTime).
  HistoryWrite bit still advertised on v1 (not knowable at materialise time) — documented.
- The update plan is evaluated before the Primary gate: an unservable request (Remove, IsDeleteModified,
  not historized, no updater) is BadHistoryOperationUnsupported off-Primary, not BadInvalidState.
- Local-kind SourceTimestamp converted to UTC before the future-skew compare and sent as UTC.
- Hosted-service wiring extracted to BuildHistoryUpdatePosture and tested (gate source, skew, clock).
- Docs: Historian.md (boot window, ordering, node-level v1, HistoryWrite bit), CLAUDE.md.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-11 11:41:11 -04:00
Joseph Doherty 93d6c1da67 fix(historian): #675 review follow-ups — SupportsReverseEvents capability, no catch-as-unsupported
- IHistorianDataSource.SupportsReverseEvents (default false; SqlServer/QuestDB true; Gateway/Null declare false).
  The node manager checks it BEFORE calling the source: an end-only capped events read becomes reverse only on
  a supporting backend (Gateway, the default, keeps the pre-#675 forward [MinValue, EndTime) paging), and an
  explicit reverse read on a non-supporting backend is BadHistoryOperationUnsupported without a call.
- Removed the broad catch (NotSupportedException) in the events path: a NotSupportedException from inside a
  backend is now logged as a backend fault (Error), not an 'unsupported' Warning.
- Null declares the reverse overload itself; guard test: every production IHistorianDataSource declares both
  the reverse overload and the capability (positive-control scan of all four backends).
- Conformance: capability reported + oversized reverse tie cluster (forward over-fetch reversed == reverse
  read order; capped reverse read stopping inside the cluster flags truncation) on SqlServer + QuestDB.
- docs/Historian.md: end-only capped events read is backward on SqlServer/QuestDB, forward on Gateway.
2026-10-11 11:36:12 -04:00
Joseph Doherty 9120182314 Merge master into feat/677-aggregates
# Conflicts:
#	docs/Historian.md
2026-10-11 11:30:46 -04:00
Joseph Doherty 27ca6cd791 feat(historian): #677 Part 13/SDK reverse anchoring, start-of-data Partial, AggregateConfiguration, IntegralDivisor
- BucketPlan: reverse processed reads anchored at the later time for every
  aggregate: window (end, start], partial interval earliest, results stamped
  with the later edge, latest first (SDK CreateSlice/GetTimestamp/backward
  GetValues). SqlServer adds @bucketAdjustTicks + flips the window predicate;
  QuestDB binds BucketBaseUnixTicks. IndexOf is the C# statement of the math.
- Partial bit on the interval the tag's data starts inside (SDK m_startOfData),
  every aggregate but Average; per-sample reads fetch the earliest sample.
- Node manager: non-default client AggregateConfiguration =>
  BadAggregateConfigurationRejected per node (ServerAggregateConfiguration,
  pinned to the SDK AggregateManager defaults).
- GatewayTagProvisioner sends IntegralDivisor = 1.
- SDK parity oracle (Opc.Ua.Server calculators) in unit + both backends'
  conformance suites, forward and reverse; remaining divergences documented.
- docs/Historian.md + CLAUDE.md release note (client-visible).
2026-10-11 11:26:00 -04:00
Joseph Doherty 8e6ac4b344 Merge feat/675-reverse-event-paging: reverse event HistoryRead paging, end-only backward, Gateway MaxValue pin (#675) 2026-10-11 11:23:51 -04:00
Joseph Doherty e59ab69398 Merge feat/674-historyupdate-posture: future-skew bound, Primary-only HistoryUpdate, SqlServer v1 accepted with modified trail disabled, QuestDB TTL pin (#674) 2026-10-11 11:21:30 -04:00
Joseph Doherty 8d154c6a81 feat(historian): #675 reverse HistoryRead-Events paging + end-only backward reads
- IHistorianDataSource.ReadEventsAsync(..., bool reverse, ...) overload: reverse = Part 11 window (lo, hi]
  newest-first by (time, id) DESC. Default forwards forward reads and throws NotSupportedException for
  reverse (Null); Gateway refuses explicitly (ascending-only stream). Server answers
  BadHistoryOperationUnsupported instead of silently returning the earliest N ascending.
- SqlServer / QuestDB serve reverse natively (ORDER BY time DESC, id DESC + TOP/LIMIT).
- Node manager: reverse event reads page with continuation points; direction stored in the cursor
  (HistoryContinuationState.Reverse) so a resume continues in the cursor's direction; reverse tie clusters
  over-fetched forward and sliced from the newest end (SliceTieCluster(reverse:)); end-only reads with a
  cap page backward from EndTime (mirrors raw).
- Resumed event pages over-fetch by the boundary skip, so every non-final page is a full cap page.
- Tests: node-manager reverse paging (ties, clusters, coarse resolution, overfetch bound, resume
  direction both ways, end-only, unsupported backend); shared EventReadConformance on SqlServer +
  QuestDB; over-the-wire reverse paging; offline Gateway pin of the UTC DateTime.MaxValue bound through
  the published client's proto conversion (in-process gRPC capture) and FILETIME.
- docs/Historian.md event paging section.
2026-10-11 11:19:54 -04:00
Joseph Doherty 0658d244d0 feat(historian): #674 HistoryUpdate posture — future-skew bound, Primary gate, SqlServer v1 accepted, QuestDB TTL pin
- HistoryUpdate items with SourceTimestamp > now + ServerHistorian:HistoryUpdateMaxFutureSkew
  (default 1 day, <=0 disables) => BadOutOfRange in the node manager (every backend, never sent);
  clock injected via OtOpcUaNodeManager.Clock (host TimeProvider).
- HistoryUpdate is Primary-only: off-Primary => BadInvalidState per node, after authorization,
  same IDriverPrimaryRoleSource verdict as the native-ack gate; HistoryRead ungated.
- SqlServer: base schema requirement back to v1; SupportsModifiedTrail (v>=2). On v1 with
  AutoCreateSchema=false: writes/reads/events/retention work, modified reads + HistoryUpdate =>
  BadHistoryOperationUnsupported with nothing sent, retention skips SampleModified, one Warning per boot.
- QuestDB integration test pinning wall-clock TTL (far-future row evicts only partitions older
  than the TTL by the wall clock), with a positive control.
- Docs: Historian.md, Configuration.md, security.md, CLAUDE.md (incl. QuestDB retry duplicates,
  #674.4 anonymous-readable trail with ACL off, #674.5 not implemented).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-11 11:18:44 -04:00
Joseph Doherty fe919f8b08 fix(client): endpoint port rewrite, Part 13 time-weighted aggregates, clean history-refusal error (#678.5, #677.5, #674.6, #678.6)
- EndpointSelector rewrites host AND port to the user-supplied URL (same scheme only)
- AggregateType/mapper/CLI/UI gain TimeAverage, TimeAverage2, Total, Total2, Interpolative
- HistoryReadCommand turns ServiceResultException into a one-line CommandException
- NodeManagerHistoryReadTests non-historized case sets IsReadModified=false
2026-10-11 10:45:18 -04:00
Joseph Doherty 7c80d36eb6 fix(historian): #668 review follow-ups (explicit modified source time, paging tests, ACL test, QuestDB orphan warning, docs)
- HistoryModifiedValue gains explicit SourceTimeUtc (SqlServer + QuestDB populate it); the node manager's resume cursor uses it instead of the prior value's nullable source timestamp (MinValue for inserts).
- ACL test now uses a trail-returning data source and asserts Good + data on the allow leg.
- Node-manager tests: end-only backward open-ended paging, reverse resume cursor, null-prior cursor.
- QuestDB updater logs a Warning (tag + time range) when a sample write fails after its trail rows landed.
- docs/Historian.md: rolling-upgrade window, IDENTITY-vs-commit ordering, QuestDB cross-node ModifiedId clock ordering.
2026-10-11 09:37:17 -04:00
Joseph Doherty 062086f9a6 Merge feat/668-modified-values: Part 11 modified-value history (IsReadModified) on SqlServer + QuestDB (#668) 2026-10-11 09:32:13 -04:00
Joseph Doherty cfd403710e feat(historian): Part 11 modified-value history on SqlServer + QuestDB (#668)
Every HistoryUpdate a SQL backend applies now leaves a modification trail
(prior value, principal, Part 11 update type, modification time), and
HistoryReadRaw with IsReadModified=true serves it as HistoryModifiedData.

- Core: HistoryUpdateContext threaded into all three IHistorianDataUpdater
  methods; IHistorianDataSource.ReadModifiedAsync (default NotSupported =>
  BadHistoryOperationUnsupported on Gateway/Null); HistoryModified* contracts.
- SqlServer: schema v2 (002_sample_modified.sql, RequiredVersion=2, docs copy);
  trail written by OUTPUT ... INTO on the same MERGE/DELETE statement, so a
  failed update leaves no audit row; retention purges SampleModified per tag on
  the source-time cutoff; ReadModifiedAsync keyed on (SourceTimeUtc, ModifiedId).
- QuestDB: append-only {prefix}sample_modified (TTL like samples), trail row
  written BEFORE the change (orphan-on-crash, never an untracked change);
  existence reads now carry the prior value; ReadModifiedAsync.
- Node manager: modified reads dispatch + page via the #669 continuation store
  (HistoryContinuationState gains HistoryCursorKind + LastModifiedId), gated by
  the existing HistoryRead decision; session principal / key id passed to the
  updater. IsDeleteModified stays unsupported.
- Client.CLI: historyread --modified.
- Tests: node-manager dispatch/paging/cursor binding/principal threading,
  over-the-wire HistoryModifiedData, shared ModifiedValueConformance on both
  backends, SqlServer atomicity (forced trail failure => no change) + v1->v2
  upgrade + retention trail purge.
- Docs: Historian.md (Modified-value history section, schema v2 upgrade note),
  security.md, Client.CLI.md, CLAUDE.md.
2026-10-11 09:26:36 -04:00
Joseph Doherty 3b739b982b Merge feat/671-time-weighted-aggregates: Part 13 time-weighted aggregates (#671) 2026-10-11 09:10:52 -04:00
Joseph Doherty 56bb4855bb fix(historian): #671 review — raw-row cap, reverse Interpolative, SDK status rule, Partial bit
M1: the time-weighted family (incl. Total, formerly a pushed-down SUM) streams
raw window rows into the C# evaluator. New shared knob
ServerHistorian:MaxTimeWeightedRawRows (default 1,000,000; <=0 unbounded with a
Validate warning) flows through HistorianBackendRegistry into both providers.
The window query is capped at cap+1 (SQL Server TOP (@maxRows), QuestDB LIMIT $4);
one row past the cap throws the new Core.Abstractions
HistoryReadLimitExceededException, which the node manager maps to
BadTooManyOperations for that node. Not counted as a backend health failure;
bound rows are not counted; per-sample aggregates are not capped.

M2: reverse Interpolative is anchored at the later time (Part 13): points at
End - i*interval, emitted latest first. 10:00->09:00 @1h yields 10:00.

L1: Partial bit (0x4, = Opc.Ua.AggregateBits.Partial) on non-Bad results of a
truncated last interval (time-weighted family; per-sample family is a follow-up).

L2/L3: aligned with the SDK AggregateCalculator.GetTimeBasedStatusCode
(decompiled from Opc.Ua.Server 1.5.378.106): TreatUncertainAsBad=false counts
Uncertain time (Uncertain samples, held-into-outage, extrapolation) as Good
time; data but no non-Bad time => generic Bad (0x80000000); BadNoData only
when there is no data at all. Consequence: a live tag's latest interval reads
Good (documented).

L5: release note no longer claims Total matches the Gateway Integral exactly
(AVEVA's divisor is unverified); it notes Total is now evaluated in C# and
subject to the cap.

Tests: unit (reverse Interpolative x3, Partial, Bad vs BadNoData incl. an
open-ended outage, cap helpers, SDK bit oracles); conformance (reverse
Interpolative, raw-row cap at/over/0, bounds not counted, health unaffected);
node manager BadTooManyOperations; options default + warning. Each new
behaviour falsified by mutation (aggregator x9, data-source cap checks on
both backends against the real DBs, node-manager catch).
2026-10-11 09:10:26 -04:00
Joseph Doherty 1f88669304 feat(historian): Part 13 time-weighted aggregates on SqlServer + QuestDb (#671)
New AggregateFamily.TimeWeighted serving TimeAverage, TimeAverage2, Total,
Total2 and Interpolative, evaluated by the shared Sql.Common
TimeWeightedAggregator for both SQL backends (identical results by
construction). HistoryAggregateType gains TimeAverage2 + Total2 (appended);
OtOpcUaNodeManager.MapAggregate maps their NodeIds. Gateway's mapper already
refuses unknown defined members => BadAggregateNotSupported per node.

Semantics (server defaults, client AggregateConfiguration ignored):
TreatUncertainAsBad=false, PercentDataBad=PercentDataGood=100,
UseSlopedExtrapolation=false. Sloped for Double/Int64, stepped for Boolean;
a usable sample followed by a Bad one is held up to the outage (AtTime rule);
stepped Uncertain extrapolation past the last sample. TimeAverage/Total use
interpolated bounding (bridge outages); TimeAverage2/Total2 exclude Bad time.
Bounding values: latest usable + latest any before start, first any + first
usable at/after end (four TOP 1 / LIMIT 1 seeks).

CLIENT-VISIBLE: Total on SqlServer/QuestDb was SUM of samples; it is now the
time-weighted integral (TimeAverage x interval seconds), matching Gateway's
Integral. Release note in docs/Historian.md.

Trade-off (deviation from the preferred hybrid pushdown, allowed by the
design): the time-weighted family is NOT pushed down. Each read streams the
window's rows plus the four bounds (SQL Server: one five-result-set batch;
QuestDB: bounds UNION ALL + window select) into the C# evaluator. Network is
O(samples in window), memory O(buckets). Writing the per-segment
interpolation/edge splitting twice in two SQL dialects and keeping them
bit-identical was judged the bigger risk; this also makes the QuestDB
lag/lead spike moot.

Tests: 35 hand-computed TimeWeightedAggregator unit cases (each of 10
implementation mutations goes red); served-aggregate pin 13 -> 17;
conformance sparse fixture (intervals wider and narrower than sample gaps)
+ an outage straddling both window edges (each of the four SQL seeks
disabled in turn goes red); node-manager NodeId mapping test.
2026-10-11 08:44:44 -04:00
Joseph Doherty 50b504d293 fix(opcua): raw open-ended/end-only reads per Part 11; extreme-bound conformance (#669) 2026-10-11 08:37:24 -04:00
Joseph Doherty acce99de9a fix(opcua): event paging review fixes (#669): open-ended end, node-bound cursors, save after projection 2026-10-11 08:30:36 -04:00
Joseph Doherty f9fd495292 fix(opcua): issue and resume continuation points for event HistoryRead (#669)
A capped events read now pages: the source's truncation flag triggers a
session-bound cursor (time inclusive + tie skip), oversized tie clusters
are paged within via a bounded half-open over-fetch, and release/unknown/
cross-kind points behave as for Raw.
2026-10-11 08:21:02 -04:00
Joseph Doherty d2be213e86 fix(historian): page by the backend's timestamp resolution; QuestDB AtTime stamps the requested time
IHistorianDataSource.TimestampResolutionTicks (default 1; QuestDB 10). A drained tie
cluster now steps off T by the resolution, so a forward one-value-per-page read on a
microsecond backend no longer re-reads the same sample forever (C1). QuestDB AtTime
results carry the client's original request time (M3); regression for the NEXT seek
skipping a tombstone (M1). Conformance backend wires the real writer and updater.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:30:25 -04:00
Joseph Doherty f8ad74dc19 fix(opcua): DeleteRawModified with an unspecified bound is BadInvalidArgument; HistoryUpdate node-manager tests
- A DateTime.MinValue start or end on DeleteRawModified is refused
  BadInvalidArgument before the backend.
- Tests: anonymous without the HistoryUpdate role denied (with it allowed),
  UpdateTagAttributes enabling historization sets HistoryWrite, a throwing
  updater is BadInternalError, the MinValue bounds case; corrected the
  test helper's comment on how the SDK base seeds unadmitted slots.
- GatewayHistorianValueWriter remarks describe the recorder's real status
  capture (and that it does not persist on the gateway).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:27:33 -04:00
Joseph Doherty 8d51380ff1 fix(opcua): reverse HistoryRead-Raw paging no longer stops after the first page
A reverse read (start > end) with a small NumValuesPerNode resumed through
the tie-cluster path, which drained the cluster and stepped FORWARD to
T + 1 tick - outside a reverse window - so the read ended with GoodNoData
after one sample. SliceTieCluster now steps down to T - 1 tick for a
reverse read (terminal at the excluded end), and the node manager slices
the over-fetched cluster in the read's own direction so the skip count is
not applied from the wrong end. Forward paging is unchanged.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:20:00 -04:00
Joseph Doherty 2ef8ee3ade feat(historian): OPC UA HistoryUpdate in the node manager + HistoryUpdate authorization (T6)
- OtOpcUaNodeManager overrides HistoryUpdateData / HistoryDeleteRawModified / HistoryDeleteAtTime,
  block-bridging to the optional IHistorianDataUpdater under the shared HistoryRead batch limiter +
  deadline. Insert/Replace/Update map to HistoryUpdateMode; Remove, IsDeleteModified, events and
  structure data are BadHistoryOperationUnsupported. Per-item codes stitched back in request order
  (MinValue SourceTimestamp => BadInvalidTimestamp, untypeable value => BadTypeMismatch); success
  clears the base's pre-seeded Unsupported error.
- HistoryWrite AccessLevel bit OR-ed in for historized nodes when an updater is wired (raw + UNS);
  it is dispatch-load-bearing in the SDK base. OtOpcUaSdkServer.SetHistorianDataUpdater; the hosted
  service assigns it from an optional HistorianUpdaterHolder before SetSink, clears it on stop.
- AuthorizeHistoryUpdate: ACL on => scoped OpcUaOperation.HistoryUpdate decision (replaces the role
  check); ACL off => #563 key re-check + new OpcUaDataPlaneRoles.HistoryUpdate role, conferred by
  ApiKeyInterimRoleMapper only from a Cluster-scoped HistoryUpdate grant (WriteOperate does not).
- AdminUI ACL editor offers NodePermissions.HistoryUpdate.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:55:42 -04:00
Joseph Doherty f7d497e532 refactor(#655): delete the dead driver-side history capability
HistoryRead is served server-side by IHistorianDataSource; nothing ever dispatched
to a driver's IHistoryProvider. Removes IHistoryProvider, the OpcUaClient history
passthrough, DriverCapability.HistoryRead (+ resilience tier rows and AdminUI row),
HistoryReadStatusException and its node-manager catches, and the tests that only
covered them. Shared result types and HistoryAggregateNotSupportedException stay
(moved to Historian/HistoryReadContracts.cs). A stored ResilienceConfig still
carrying a HistoryRead key parses (diagnostic only; new test).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 23:34:02 -04:00
Joseph Doherty f02067d295 test: retry OPC UA host start on a port bind race (TOCTOU in AllocateFreePort)
The tests probe a free port (bind 0, release) and start the SDK server later; under a
highly parallel run another test can take the port in between and the SDK fails with
'Failed to establish tcp listener sockets for Ipv4 and IPv6.'. OpcUaHostStart.StartAsync
builds a fresh host + server on a fresh port per attempt (up to 5) when the failure is a
bind failure; all 45 start sites in OpcUaServer.Tests, OpcUaServer.IntegrationTests and
Runtime.Tests use it. Per-attempt GUID ApplicationUris are hoisted out of the host factory
because a changed URI against a reused PKI store fails with 'Cannot access private key'.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 22:30:55 -04:00
Joseph Doherty b8c2136732 fix(opcuaclient): #649 review minors - CodeBits match, Bad-only exception ctor, BadNoData + Raw-resume tests, latent-impact docs
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 19:29:47 -04:00
Joseph Doherty 17dc171940 fix(opcua): carry the condition Comment across a rebuild; the restore report never rolls Time back (#641 review)
Native conditions support AddComment / commented Acknowledge, so the comment (and its
ClientUserId) is condition state the rebuild dropped. And the deferred restore report
re-stamped Time with the carried instant even when a client method call had reported a
newer event in the restore->complete window; it now reports at the node's current Time.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 17:20:13 -04:00
Joseph Doherty ddd7678269 fix(opcua): carry native alarm condition state across a full rebuild (#641)
A full RebuildAddressSpace recreated every native Part 9 condition inactive +
acknowledged, and when the alarm-tag set was unchanged (e.g. one
UnsTagReference added to an alarm-bearing raw tag) nothing re-told the new node
its state: the host does not re-subscribe for an unchanged set, and Galaxy /
OpcUaClient have no snapshot to replay anyway. Active alarms read normal until
their next device transition.

OtOpcUaNodeManager now captures each NATIVE condition's live state (from the
node, so a client-applied ack is kept) before the rebuild clears, restores it
when the same id is re-materialised native in the same apply, and reports ONE
condition event per restored off-normal condition at the new
IOpcUaAddressSpaceSink.CompleteAddressSpaceApply (after notifier wiring).
Unclaimed captures are discarded there, so a later re-add is fresh. Quality is
carried silently for normal conditions too (it is only re-annotated on a
connectivity edge). Scripted conditions are untouched (#487 re-asserts them).
OpcUaPublishActor calls CompleteAddressSpaceApply at the end of every apply.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 17:02:12 -04:00
Joseph Doherty aba3aa7a8d fix(opcua): condition Severity is carried verbatim, not re-bucketed to 5 points
OtOpcUaNodeManager.MapSeverity re-bucketed every projected severity onto the
SDK's five EventSeverity values (<800 -> 700, else 900), defeating #626's exact
OpcUaSeverity: an MTConnect Fault (800) read and evented as 900; a scripted
alarm graded 1000 published 900 while /alerts said 1000; a native Low (200)
published 300. EventSeverity merely names points on the 1..1000 scale, so the
node now carries the value clamped to 1..1000 (ToPart9Severity), at
materialise, projection and in the delta gate.

MTConnect wire test now asserts Fault ActiveState=true/Severity=800 and the
Normal clear ActiveState=false/Retain=false/Severity=800.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 16:19:27 -04:00
Joseph Doherty 55eac5f68b feat(history): full Part 13 aggregate set; unsupported-by-backend aggregates answer BadAggregateNotSupported (#285)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 14:24:25 -04:00
Joseph Doherty 52708bff52 fix(address-space): numeric array writes convert (checked) to the node element type; array-to-scalar refusal only for Modbus/S7 (#610 review R6)
MAJOR-1: R4's exact element-type match refused ScadaBridge's CLR-list
writes (its element types are only Boolean/Int32/Float/Double/String/
DateTime) to every Int16/UInt16/Byte/UInt32/Int64/UInt64 array node and a
Double list to a Float array. EvaluateEquipmentWriteStructure now converts
a numeric array of another numeric element type element-wise, CHECKED, into
the node's element type and hands the converted array back through
OnEquipmentTagWrite's ref value, so the node stores and the driver receives
the node's own type. Any out-of-range, fractional-to-integer, NaN/Inf-to-
integer or Float-overflowing element refuses the whole write BadTypeMismatch
(nothing dispatched) - so the UInt32 -> Int32 bit reinterpretation stays
closed. Non-numeric mismatches stay refused. The wrong 'no client relies on
a looser rule' comment is corrected.

MINOR-1: the array-value-to-scalar-node refusal broke writes back to
browse-committed array tags (committed scalar, isArray never set) on
pass-through drivers. It now applies only when the owning driver's scalar
encoder cannot take an array (DriverArrayWriteSupport.
RefusesArrayValueOnScalarTag: Modbus, S7), carried RawTagPlan.
ScalarRefusesArrayValue / UnsReferenceVariable.BackingScalarRefusesArrayValue
-> EnsureVariable(refuseArrayOnScalar) -> a per-node set in the node
manager. OpcUaClient/Galaxy/others keep pre-#610 dispatch. Not on the
classifier whitelist (driver-type edit rebuilds); a mirror whose flag
changes is recreated, not updated in place.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 08:31:46 -04:00
Joseph Doherty a702875d41 Merge review R4 fixes (#610): array writes require an exact element-type match (BadTypeMismatch, nothing sent); Galaxy encoder refuses uint[]/ulong[]; Modbus array writability bounded by maxRegistersPerWrite/coil cap; arrayLength >= 1 required; Galaxy arrays without a declared length stay read-only; arrays to scalar nodes refused
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 08:08:03 -04:00
Joseph Doherty dabd8eb955 fix(address-space): exact element-type check for array writes + array-capability shape edges (#610 review R4)
MAJOR-1: the SDK skips TypeInfo.IsInstanceOfDataType on any node with an
OnWriteValue handler (every writable equipment node), and the node manager's
fail-fast treated numeric<->numeric as compatible, so a UInt32[] to a Galaxy
Int32 array matched MxValueEncoder's int[] arm by CLR covariance and was
written bit-reinterpreted with a Good result. EvaluateEquipmentWriteStructure
now matches an ARRAY write's element type exactly (Byte[]/ByteString the only
equivalence; Variant[]/object[] refused) -> BadTypeMismatch, nothing
dispatched. MxValueEncoder's int[]/long[] arms are exact-type guarded too, and
the Galaxy parity test's false "the SDK refuses" premise is corrected (the
UInt32/UInt64 exclusion is now the encoder's own answer). Scalar writes keep
numeric widening by decision: consumers write CLR-typed values (Double to a
Float node) and every driver's scalar encoder range-checks.

MINOR: Modbus row honours maxRegistersPerWrite (smallest of DriverConfig /
the tag's DeviceConfig, default 123) and FC15's 1968-coil ceiling, and
requires region + numeric address; Modbus/S7/Galaxy require isArray + integer
arrayLength >= 1 (Galaxy: an unbounded node would let a short write resize the
attribute); an array value to a scalar node is refused inline. Docs: Uns.md
array-write rules + upgrade note (ReadWrite array tags become client-writable
on first boot), Modbus.md, Galaxy.md.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 08:07:36 -04:00
Joseph Doherty b99aea46b2 Merge review R1 fixes: removed driver's resilience tracker released; SDK timed-unshelve no-op (engine owns expiry); commented re-ack stored as AddComment; native-ack Primary verdict published from start; drain gate SharesRowsWithPeer; malformed cache bytes don't adopt the revision; HostAbortedException passthrough; ClusterRoleInfo seed filtered
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 07:47:57 -04:00
Joseph Doherty 83e9ffd27a fix(alarms): SDK timed-unshelve timer is a no-op; commented re-ack comment is durable (#597)
Review R1 M2: since #597 the SDK timer fires at real expiry, and OnTimedUnshelve routed an operator-style
Unshelve by 'system' from both pair nodes, pre-empting the engine's AutoUnshelve. The engine owns expiry,
so the callback now routes nothing. Integration test: an engine stand-in that acts on a routed Unshelve and
expires late, plus a wrapped OnTimedUnshelve (positive control: the SDK timer fired).
Review R1 m1: Part9StateMachine.ApplyAcknowledge records a commented re-ack as AddComment (persisted and
replicated) instead of a NoOp; an uncommented re-ack is still a no-op.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 07:47:28 -04:00
Joseph Doherty 18f8eb5d2f feat(address-space): server-side array writes gated per driver capability (#610)
Array nodes were forced read-only for every driver. A new single authority,
Core.Abstractions DriverArrayWriteSupport, says which (driver, element type)
pairs encode a whole-array write: Modbus (Coils / numeric HoldingRegisters),
S7 (Byte/Int16/UInt16/Int32/UInt32/Float32), OpcUaClient (pass-through) and
Galaxy (Boolean/Int32/Int64/Float/Double/String/DateTime). Everything else
stays read-only, so the server never advertises a write the driver refuses.

- Composer derives RawTagPlan.ArrayWritable + UnsReferenceVariable
  .BackingArrayWritable at compose time (both seams, byte-parity); the
  applier's one writability rule is Writable && (!IsArray || ArrayWritable)
  for materialise and surgical paths, raw and UNS alike.
- ArrayWritable is on the classifier's surgical raw-tag whitelist and
  mirrored on the UNS record (UnsReferenceBackingParityTests updated).
- Node manager refuses a wrong-length / non-array value on an array node
  (BadTypeMismatch) and any IndexRange write (BadWriteNotSupported) inline,
  before dispatch; WriteOperate/ACL gates and the revert are unchanged.
- Per-driver parity tests pin the table against the real Modbus write path,
  S7 EncodeArrayBlock and Galaxy MxValueEncoder (UInt32/UInt64 excluded:
  array covariance reinterprets their bits).
- Over-the-wire proof in OpcUaServer.IntegrationTests (ArrayWriteWireTests).
- ValueRank > 1 documented as out of scope by decision (docs/Uns.md).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 07:39:05 -04:00
Joseph Doherty ca0f1e7735 Merge #597/#599: real UnshelveTime, commented re-ack reports, native ack refused off-Primary, /alerts ack-path test; surgically-added raw nodes announced, equipment-with-references removal keeps references severed; Backing* parity guard
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 07:12:59 -04:00
Joseph Doherty 0bd3ec00e4 fix(address-space): announce surgically-added raw nodes; equipment-with-references removal stays surgical (#599)
- AnnounceAddedNodes raises NodeAdded in the Raw realm for the existing parents of
  added raw containers/tags (ComputeRawAddAnnouncements); Uns side unchanged.
- ApplyPureRemove removes UNS reference variables BEFORE the equipment subtree, so a
  reference the subtree's id-prefix scope reaches no longer ratchets to a full rebuild
  (03/P5). A NotSubsumed filter was not used: in production a reference NodeId is
  Area/Line/Equipment/Name, outside the subtree scope, so filtering would leak it.
  The matching RemoveEquipmentSubtree severing (03/S20) is in the node-manager commit.
- UnsReferenceBackingParityTests: reflection guard discovering the classifier's surgical
  raw-tag whitelist and requiring a same-typed mirror field (03/U11).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 07:12:29 -04:00
Joseph Doherty dc54b883eb fix(historian): provision tags whose historization flips on or historian name changes (#600)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 06:45:48 -04:00
Joseph Doherty 167a48bffa fix(alarms,acl): one condition event per Part 9 method call; gate Server-object model-change/audit events (#587, #588)
#587 — a client Acknowledge/Confirm/AddComment/Shelve/Enable/Disable emitted no condition event: the SDK's
ReportStateChange is gated on AutoReportStateChanges (never set), and the engine re-projection found the node
already in the new state and was suppressed by the delta-gate. WireMethodTransitionReporting now wraps each
method's SDK handler and, on Good, reports exactly once via ReportConditionEvent, under Lock so the routed
re-projection cannot race in ahead of the SDK applying the state. Independent of the SDK's imprecise
AreEventsMonitored bookkeeping. The shelve methods' state is applied by the wrapper (the SDK never applies
it). Native conditions' Confirmed/Shelving are method-owned (NativeAlarmProjector's constants would re-report
every ack and undo every shelve). MaxTimeShelved set to int.MaxValue ms at materialise — Create() left it 0
and every TimedShelve failed BadShelvingTimeOutOfRange. Wrong "SDK auto-fires (E2)" comments fixed.

#588 — with ACL on, GeneralModelChange and AuditWriteUpdate events no longer go to Server.ReportEvent
(ungated to every i=2253 item): ReportToServerObject delivers them per item, a model change narrowed to the
Affected nodes the session may Browse (dropped when none), an audit write only with Read on its source plus
the SDK audit/encryption guard. Flag off: unchanged.

#586 review: materialise raises only the new condition's monitored unit (was a full O(N) reconcile per
condition ⇒ O(N²) deploys under Lock).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-08 00:54:08 -04:00
Joseph Doherty 2b945672e9 fix(acl): revalidate event/alarm subscriptions on both triggers; key-usability check on role gates (#562, #563)
#562: direct event (alarm) monitored items are now tracked in a second map
(_eventItems, fed by the SubscribeToEvents override) and re-authorized by
RevalidateMonitoredItems on both triggers (DataPlaneAuthorityState.Changed and
AclRevalidationTimer). A denied item is detached via the SDK's own protected
SubscribeToEvents(unsubscribe: true) - never MonitoringMode.Disabled - and
re-attached on restore; a client delete of a stood-down item is absorbed.
Over-the-wire tests: revoke -> stop (positive control on a still-granted
notifier) -> restore -> resume; clock-driven expiry via the real timer with a
control session; delete-while-stood-down is not resurrected; unexpired
negative control. Server-object (all-events) subscriptions are documented as
outside the ACL (observed over the wire).

#563: DataPlaneAuthorizer.AuthorizeKeyUsable runs the FindUsableKey probe on
the WriteOperate and AlarmAck role gates regardless of Security:Acl:Enabled,
so an expired or deployment-disabled key loses its activation-time roles on
its open session. Anonymous (no key id) is unaffected; reads/subscriptions
stay ungated with the flag off. No new flag.

Docs: docs/security.md and the CLAUDE.md API-key/ACL section.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-07 23:10:35 -04:00
Joseph Doherty 1d251cc037 Merge branch 'master' into fix/r3-02-stale-good-family
# Conflicts:
#	src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs
#	tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Harness/V3ArtifactSeeder.cs
2026-08-20 11:03:02 -04:00
Joseph Doherty ec5b48bfcd feat(opcua): IOpcUaAddressSpaceSink.WriteStatuses — status-only batch write retaining the node value (03/S15) 2026-08-20 10:00:47 -04:00
Joseph Doherty b09f78396f test(opcua): pin clock-driven expiry in the revalidation pass; correct the only-trigger remark 2026-08-20 09:50:39 -04:00
Joseph Doherty 89630ee418 fix(acl): browse transit reaches the shared root folder (narrow Browse grants were unreachable)
With Security:Acl:Enabled=true, a session holding any grant narrower than
Cluster could not browse the served address space at all — confirmed by the
live gate.

Mechanism. Both realm trees hang under ONE anchor folder minted in
CreateAddressSpace: NodeId("OtOpcUa", <raw ns>). ResolveParentFolder parents
every top-level raw folder AND every top-level UNS area under it, and the
Browse override throws BadUserAccessDenied for the WHOLE call when its source
node is denied. DataPlaneAuthorizer.BuildScope split that identifier like a
path, producing {Kind: Raw, Segments: ["OtOpcUa"]} — but trie grant paths are
["raw", <RawPath segments…>] and "OtOpcUa" is never a path segment, so
PermissionTrie.HasBrowseTransit failed at the realm-child lookup and
CollectMatches yielded only Cluster-scoped grants. Deny, at the front door.

The trie already handled a zero-segment scope as "the realm root, itself an
ancestor of everything granted in that realm" (PermissionTrie.cs:101-118) —
no production path had ever produced that scope. This wires it up.

- V3NodeIds.SharedRootFolderId names the anchor identity; CreateAddressSpace
  now mints the NodeId/BrowseName/DisplayName from it (behaviour-identical,
  makes the coupling explicit).
- BuildScope maps that identifier (realm-qualified to Raw — in the UNS
  namespace s=OtOpcUa is an ordinary Area) onto the zero-segment Raw scope.
- Evaluate adds a second-realm OR-leg for Browse on the anchor only: it fronts
  both trees, so a principal whose only Browse grant lives in the UNS tree
  still enters through this raw node. Browse only; transit confers navigation,
  never read/write/subscribe.
- No trie, evaluator or builder change. No reordering above the decision cache
  — key revocation is still re-checked before the lookup — and the cache key
  is unchanged (the combined verdict caches under the anchor's own
  identifier/realm; a generation advance clears it).

Falsifications actually run (implement-then-revert-the-line, plus one
genuine red-first):

- DataPlaneAuthorizerTests: the 7 new facts were written BEFORE the fix and 4
  went red (the 3 deny-only ones passed vacuously, as a deny-only test must).
- Deleting the UNS OR-leg reddens exactly one fact,
  The_shared_root_folder_is_browsable_on_a_uns_only_browse_grant (26/27 pass).
- Dropping the realm qualification from IsSharedRootFolder reddens exactly
  A_uns_area_named_like_the_anchor_gets_no_anchor_semantics. Its first draft
  did NOT catch this — a raw-only grant is refused either way; the
  discriminating grant had to be an unrelated UNS one, whose own transit marks
  the zero-segment UNS scope the un-qualified mapping would land on.
- Disabling the BuildScope anchor branch reddens the new end-to-end fact
  NodeManagerStage6EnforcementTests.The_shared_root_folder_is_browsable_on_a_narrow_grant_and_still_filters
  (real booted server, real Browse dispatch: BadUserAccessDenied on the whole
  call), which also pins that the reference filter still withholds the
  non-granted sibling.
- Core PermissionTrieBrowseTransitTests gains 3 characterization facts pinning
  the previously-dead realm-root branch (transit true for the granting realm's
  root, false for the other realm's, false without the Browse bit). These were
  green on arrival — the trie was always correct; nothing called it.

Tests: Core 351 -> 354 passed; OpcUaServer 482 -> 490 passed. Build clean.
2026-08-18 09:45:04 -04:00
Joseph Doherty e29fab19f6 test(acl): correct the array-cell comment — captured locals do take ref (re-review nit) 2026-08-18 07:13:16 -04:00
Joseph Doherty f4b91dbbab test(acl): race harness — hit diagnostics, 10k rounds, idiom notes (review round) 2026-08-18 07:10:44 -04:00
Joseph Doherty 441b34a391 fix(acl): make InvalidateIfStale a monotonic-max CAS loop — stamp publication is never deferred 2026-08-17 07:15:30 -04:00
Joseph Doherty d54b4fab62 docs(acl): fix the two comment nits from the T1 re-review — displacement precision, straggler-test claim 2026-08-17 06:09:13 -04:00