PR #423 shipped GatewayTagProvisioner + unit tests but never registered it in
DI nor passed it into the AddressSpaceApplier, so deploying historized tags used
the no-op NullHistorianProvisioning and never called the gateway's EnsureTags
(confirmed live on wonder-app-vd03: zero EnsureTags calls on a historized deploy).
Addresses HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md.
Issue 1 (wire provisioner):
- Runtime: AddHistorianProvisioning extension (gated on ServerHistorian:Enabled,
mirrors AddServerHistorian) + NullHistorianProvisioning TryAdd default in
AddOtOpcUaRuntime; WithOtOpcUaRuntimeActors resolves IHistorianProvisioning and
passes it into the applier.
- Gateway driver: GatewayHistorian.CreateProvisioner factory (mirrors CreateDataSource).
- Host: Program.cs calls AddHistorianProvisioning after AddServerHistorian.
- Tests: AddHistorianProvisioningTests (config-gated registration + the
register->resolve->applier->EnsureTags chain).
Issue 2 (observability): AddressSpaceApplier logs the provisioning tally on every
successful dispatch (was gated behind Failed/Skipped > 0), including dispatched=N
so a dispatched=N/requested=0 line flags the dormant no-op. +2 tests.
Issue 3 (30s HistoryRead on unprovisioned tags): root coupling fixed by Issue 1;
documented the CallTimeout knob + coupling. Default left at 30s pending the
multi-data-point investigation the issue requests (lowering risks truncating
legitimate large reads).
Issue 4 (docs): docs/Historian.md gains a "Tag auto-provisioning (EnsureTags)"
section and CLAUDE.md a wiring/gating note (both stress ServerHistorian:Enabled).
Sibling scadaproj/CLAUDE.md carries no false claim -> unchanged.
Pre-existing Serilog observation: anchor CWD to AppContext.BaseDirectory before
AddZbSerilog so the relative file sink stops landing in C:\Windows\System32 under
the Windows-service CWD.
Builds 0-error; Runtime.Tests 355, OpcUaServer.Tests 329, Gateway.Tests 99 (+4
live-skipped) all green.
The continuous-historization recorder conflated two identifiers into one string:
the dependency mux fans DependencyValueChanged keyed by the driver FullName
(the mux ref), but a value must be historized under the resolved historian name
(HistorianTagname override, else FullName). In the common no-override case the
two are equal, so it worked; with an override they diverge and the recorder
registered mux interest under a key the mux never fans — that tag's values were
never captured (and, had they been, would have been written under the mux ref).
Carry BOTH identifiers through the seam: a new HistorizedTagRef(MuxRef,
HistorianName) record on IHistorizedTagSubscriptionSink. The applier resolves
MuxRef = FullName and HistorianName = override-or-FullName. The recorder now
keeps a muxRef->historianName map: it registers/filters mux interest by MuxRef
but writes the outbox entry (and drains) under HistorianName. The convergence
handler re-registers the mux only when the registered key-set changes, so an
override-only rename (same FullName) updates the write target without mux churn.
Tests: a divergent-override recorder test (interest by mux ref, value written
under the override name, never the mux ref) + an override-rename no-churn test;
the applier feed tests now assert the full (mux ref, historian name) pairs.
Runtime 348/0, OpcUaServer 327/0; 0 warnings. Closes FU-3.
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
The ContinuousHistorizationRecorder was spawned with an EMPTY historized-ref
set, so it registered interest in nothing and historized nothing. This feeds it
the currently-historized tag refs on every address-space deploy/redeploy so its
DependencyMuxActor interest converges to exactly the historized set (the same
refs the EnsureTags provisioning hook resolves: override-or-FullName).
Design — delta convergence (the plan is a pure DIFF):
- New seam IHistorizedTagSubscriptionSink (Core.Abstractions/Historian) with a
Null no-op singleton, mirroring how IHistorianProvisioning decouples the T15
hook. AddressSpaceApplier gains a DEFAULTED ctor param (Null sink) so all ~80
existing call sites + the production site compile unchanged.
- Apply() only ever sees a plan diff (an incremental/surgical apply carries a
delta, not the full set), so the applier feeds an add/remove DELTA computed
from AddedEquipmentTags / RemovedEquipmentTags / ChangedEquipmentTags. The
recorder keeps the full set and re-registers it. The feed is a single
non-blocking Tell behind the sink, wrapped in try/catch so a faulting feed
never blocks or breaks a deploy (same discipline as the provisioning hook).
- Recorder.UpdateHistorizedRefs(added, removed) converges the tracked set, then
— only when it actually changed — sends ONE RegisterInterest with the full set
(the mux's RegisterInterest is a full-REPLACE) or one UnregisterInterest when
it drains to empty (the mux has no per-ref unregister). An unchanged delta is
a no-op (no mux churn).
- DI: the recorder is now spawned BEFORE the applier so the adapter
(ActorHistorizedTagSubscriptionSink) can wrap its IActorRef; the Null sink is
used when continuous historization is off/unwired.
Tests: recorder convergence (add-from-empty, add+remove converge, idempotent,
drain-to-empty unregisters); applier feeds resolved added refs, removed+renamed
deltas, and survives a throwing sink. Build clean (0 warnings on touched
projects); Runtime/OpcUaServer/Gateway/AdminUI suites green.
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
HistorianGateway is now the sole historian backend (read + alarm SendEvent +
continuous WriteLiveValues). Document the final state and retire the Wonderware
sidecar from the docs/config/labels:
- CLAUDE.md: rewrite the Historian section — ServerHistorian /
ContinuousHistorization / AlarmHistorian config keys, the IHistorianProvisioning
EnsureTags hook, the GatewayAlarmHistorianWriter SendEvent path + ReadEvents
dependency on gateway RuntimeDb:EventReadsEnabled=true, gateway-side
prerequisites (RuntimeDb flags + historian:read/write/tags:write scopes),
migration note, and two KNOWN-LIMITATION callouts (live-validation gate +
empty historized-ref-set recorder follow-on).
- appsettings.json: fix the stale ServerHistorian block (Host/Port/SharedSecret/
ServerCertThumbprint -> Endpoint/ApiKey/UseTls/AllowUntrustedServerCertificate/
CaCertificatePath/CallTimeout, keep MaxTieClusterOverfetch); add a disabled
ContinuousHistorization block; prune the orphaned Wonderware keys from
AlarmHistorian (keep the SQLite knobs). ApiKey env-supplied via
ServerHistorian__ApiKey (commented; valid strict JSON via _comment keys).
- README.md + docs (Historian.md, AlarmHistorian.md, Configuration.md,
ServiceHosting.md, DriverLifecycle.md, drivers/README.md, Uns.md, VirtualTags.md,
AlarmTracking.md, Client.UI.md, README.md, TestConnectProbes.md): retire the
Wonderware historian backend from current-backend descriptions; fix the stale
ServerHistorian/AlarmHistorian config tables (now gateway shape); convert
drivers/Historian.Wonderware.md to a retired stub pointing at the gateway.
- Source/UI labels (descriptive text only, no behavior change):
OtOpcUaServerHostedService.cs, HistoryPaging.cs, OtOpcUaSdkServer.cs,
HistorianAdapterActor.cs, VirtualTagModal.razor, ScriptedAlarmModal.razor,
AlarmsHistorian.razor now name the HistorianGateway backend.
Build clean (0 errors); AdminUI.Tests green (514 passed).
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
Addresses T15 review: treat a canceled EnsureTags task like a faulted one so the
fire-and-forget continuation never reaches t.Result (which would re-throw and leave
the discarded task unobserved).
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
A rename-only deploy produced an IsEmpty plan that short-circuited before MaterialiseHierarchy,
leaving the OPC UA folder DisplayName stale. AddressSpacePlanner now diffs UnsAreas/UnsLines by
stable id into a RenamedFolders set (counted in IsEmpty); the applier refreshes the folder in
place via a new UpdateFolderDisplayName on ISurgicalAddressSpaceSink (forwarded through
DeferredAddressSpaceSink so it is NOT inert on driver hosts; falls back to rebuild when the sink
is non-surgical). DeploymentArtifact byte-parity untouched (rename rides the existing Name
round-trip). No EF migration, no serialized wire/proto contract change. +13 OpcUaServer tests, Runtime rebuild test.
- OpcUaServer-002: HistoryRead-Events NumValuesPerNode==0 now maps to unbounded (int.MaxValue) instead of the backend default-cap sentinel; no Core.Abstractions contract change (+EventMaxEvents helper tests)
- OpcUaServer-004: EnsureAddressSpaceCreated guard on public mutators -> clear InvalidOperationException instead of bare NRE if called pre-start (+tests)
- OpcUaServer-003: Deferred (endUtc inclusive/exclusive needs live Wonderware boundary confirmation)
- Configuration-013: wire DraftValidator.ValidateClusterTopology into AdminOperationsActor deploy gate (read-only, no migration) (+2 tests)
Cross-module fix from the review sweep. -007 (Medium): OnTimedUnshelve built its AlarmCommand
with User=string.Empty, so Part9StateMachine.ApplyUnshelve rejected it (ArgumentException,
swallowed) and a TimedShelve never auto-expired. Pass the canonical 'system' user; the
AlarmAck-gate bypass is preserved. Repurposed the test that had encoded the bug.
Live-verify finding: the surgical DataType/array path only fires for drivers whose
TagConfig carries a stable top-level FullName (Galaxy/OpcUaClient). For protocol
drivers (Modbus/S7/...), ExtractTagFullName falls back to the raw TagConfig blob, so
a shape edit also mutates FullName → safe full-rebuild fallback. Comment-only.
Code-review follow-ups on the FB-7 surgical shape-write commit:
- GeneralModelChangeEvent now sets SourceNode=Server + SourceName (Part 3 §8.7.4)
so clients filtering events by SourceNode match it (report still uses source:null).
- UpdateTagAttributes adds an explicit dataType null/empty guard (widened surface).
- Tighten the ArrayLengthDiffers doc comment.
- Add array→scalar transition test + null-arrayLength zero-default test (coverage
symmetry). 275/275 OpcUaServer.Tests green.
Widen the F10b surgical address-space path so a changed equipment tag whose
only differences are DataType / IsArray / ArrayLength (on top of the existing
Writable / Historizing) is applied IN PLACE on the live node instead of forcing
a full RebuildAddressSpace that drops every client's subscriptions server-wide.
ISurgicalAddressSpaceSink.UpdateTagAttributes gains (dataType, isArray,
arrayLength); the DeferredAddressSpaceSink wrapper forwards all six args (the
prod-inertness seam). OtOpcUaNodeManager swaps DataType + ValueRank +
ArrayDimensions in place, and on a real shape change (a) resets the node to
BadWaitingForInitialData so no stale wrong-typed value is exposed (closes the
prior brief-value-type-mismatch objection) and (b) raises a Part 3
GeneralModelChangeEvent (verb=DataTypeChanged) so model-aware clients re-read
the definition. A Writable/Historizing-only change leaves the shape untouched
(no reset, no model event) — original behaviour preserved byte-for-byte.
AddressSpaceApplier.TagDeltaIsSurgicalEligible adds the three shape fields to
its whitelist; FullName/Name/DriverInstanceId/alarm differences still rebuild.
Tests: new NodeManagerSurgicalShapeUpdateTests boots a real server to prove the
in-place swap + value reset + the no-reset backward-compat path + the model-event
builder; AddressSpaceApplierTests invert the two former DataType/IsArray-rebuild
cases to surgical and assert the shape args land; DeferredAddressSpaceSinkTests
assert the shape args forward. 273/273 OpcUaServer.Tests green; full solution builds.
The OPC UA address-space build pipeline was named after a v2-roadmap
milestone number rather than its domain. Rename the family to describe
what it does (build/diff/apply the OPC UA address space):
Phase7Composer -> AddressSpaceComposer
Phase7CompositionResult -> AddressSpaceComposition
Phase7Planner -> AddressSpacePlanner
Phase7Plan -> AddressSpacePlan
Phase7Applier -> AddressSpaceApplier
Phase7ApplyOutcome -> AddressSpaceApplyOutcome
The 9 Phase7*Tests suites follow suit; Phase7ScriptingEntitiesTests ->
ScriptingEntitiesTests (it tests the scripting migration, not the
pipeline). Log-message prefixes move to the new class names.
Pure mechanical rename, no behavioral change. EF migration classes/IDs
(AddPhase7ScriptingTables, ExtendComputeGenerationDiffWithPhase7) are
immutable and left untouched, as are historical design docs.
Build clean; OpcUaServer 261/261, Runtime 272/272, ScriptingEntities
12/12 green.
Extends RecordingSink to capture isArray/arrayLength per EnsureVariable call,
adds two applier-level tests asserting the wire-through for array and scalar
plans, adds float/overflow InlineData rows to ExtractTagArray theory, and
corrects the ExtractTagArray XML-doc wording (null => unbounded ArrayDimensions=[0]).
C1 (critical): a boundary tie cluster larger than NumValuesPerNode could
silently truncate a resumed read to GoodNoData, permanently dropping the
un-emitted ties — the (timestamp, skip) cursor cannot advance past a single
timestamp the fixed-(start,end,cap) backend keeps re-returning. Now detected
and failed LOUDLY per node with BadHistoryOperationUnsupported + a log naming
the tag/timestamp/cap; documented in Historian.md with the larger-cap remedy.
Regression test Raw_tie_cluster_larger_than_page_fails_loudly_not_silently.
I3: build HistoryData before Save() so a projection failure can never orphan a
stored continuation cursor.
N1 (YAGNI): drop the never-produced HistoryReadKind enum + Processed-only
Aggregate/IntervalTicks fields from HistoryContinuationState — only Raw pages.
N3: ComputeResumeCursor guards its documented non-empty precondition.
I1: document InMemoryHistoryContinuationStore's eventual-consistency (test double).
Build clean, 182/182 OpcUaServer tests pass.
The Wonderware historian backend is single-shot — it returns up to
NumValuesPerNode samples with a null continuation point — so paging is
synthesised server-side, time-based, for the only count-capped arm (Raw):
- A full page (count == NumValuesPerNode, NumValuesPerNode > 0) emits an
opaque 16-byte continuation point and stores a resume cursor; a short page
(or NumValuesPerNode == 0 "all values") emits none.
- A resume read takes the stored cursor, reads the next page from the boundary
forward, and emits a fresh CP only if that page is also full.
- The resume cursor is tie-safe (HistoryPaging.ComputeResumeCursor /
TrimBoundaryDuplicates): the next page resumes from the boundary timestamp
INCLUSIVE and drops the head ties already returned, so samples sharing the
boundary SourceTimestamp are neither duplicated nor skipped.
Continuation points are bound to the OPC UA session via the SDK's
ISession.SaveHistoryContinuationPoint / RestoreHistoryContinuationPoint store
(SessionHistoryContinuationStore) — capped by ServerConfiguration.
MaxHistoryContinuationPoints (default 100, oldest-evicted) and disposed on
session close. releaseContinuationPoints is honoured via an override of
HistoryReleaseContinuationPoints (the base dispatcher routes release-only reads
there, never to the per-details arms). An unknown / evicted / released point
resumes to BadContinuationPointInvalid.
Processed and AtTime stay single-shot: neither details type carries a client
count cap, so the single-shot backend returns the complete result in one read
and there is no "full page" signal to page on (spec-conformant). Modified-value
history remains out of scope.
The pure paging decisions + CP store contract are unit-tested via HistoryPaging
+ InMemoryHistoryContinuationStore; the full multi-page round trip is driven
end-to-end through the node manager with an in-memory store + a series-backed
fake historian (the in-process harness is session-less).
- A.1 (false-rejection safety): restrict the structural fail-fast's confident-mismatch check to
the CLOSED set of built-in types ResolveBuiltInDataType emits (numeric families + Boolean/
String/DateTime/ByteString). Any other expected type (Enumeration, Guid, …) now defers to the
SDK, so a coercible write (Int32→Enumeration) is never false-rejected. + A7/A8 regression tests.
- C.1: guard BuildWriteFailureAuditEvent (under Lock) in try/catch like ReportAuditEvent, so a
SetChildValue surprise is swallowed+logged, never thrown out of the fire-and-forget continuation.
Three deferred 'surface the failed write' enhancements on the write-outcome
self-correction path in OtOpcUaNodeManager:
- Item A: synchronous structural fail-fast. EvaluateEquipmentWriteStructure
(pure static) rejects a structurally-invalid write INLINE (Bad sync) after
the authz gate but before the optimistic dispatch, so the SDK never applies
it. Null payload -> BadTypeMismatch; plus a confidence-gated cheap built-in
type compatibility check (numeric widening + BaseDataType wildcard tolerated;
uncertain cases defer to the SDK's own coercion).
- Item B: Bad-quality blip on device-write failure. On a revert,
RevertOptimisticWriteIfNeeded first publishes the still-applied optimistic
value with StatusCode BadDeviceFailure, then restores the prior value/status
(both under the existing Lock). Documents the queue-coalescing caveat (a slow
subscriber may see only the restored value -> the audit event is the reliable
signal).
- Item C: Part 8 AuditWriteUpdateEvent on device-write failure. Builds an
AuditWriteUpdateEventState (SourceNode=node, AttributeId=Value, OldValue=prior,
NewValue=attempted, ClientUserId from the threaded identity, Message carries
outcome.Reason) under Lock and reports it via Server.ReportEvent OUTSIDE Lock.
Guarded so auditing-disabled / report failure never breaks the revert.
Threads the writing identity's user-id + node into the continuation. Adds 6
unit tests for EvaluateEquipmentWriteStructure. Build clean (0 warnings);
158/158 OpcUaServer.Tests green.
- HistoryReadEvents miss path + catch path now both set results[handle.Index] explicitly
(new SdkHistoryReadResult { StatusCode = BadHistoryOperationUnsupported }) — don't rely on
base pre-seeding results[i] so every path sets BOTH errors and results coherently (#1)
- ProjectEventField: SourceName null now emits Variant.Null instead of a String-typed null
variant (evt.SourceName is null ? Variant.Null : new Variant(evt.SourceName)) (#3)
- Comment near the HistoryRead dispatcher block updated: all four arms (Raw/Processed/AtTime
+ Events/Task 4) are now overridden — "left to the base" wording was stale (#5)
- Happy-path test adds ReceiveTime to select clauses and asserts it projects ReceivedTimeUtc
as a DateTime Variant at the correct select-order position (#4)
- Backend-throw test hardened: asserts errors[0] via ServiceResult.IsBad + explicit code,
asserts results[0] is non-null with the Bad code (no longer relies on base seeding),
and asserts EventsEntered to prove the override reached the bridge before the throw (#1)
- RecordingHistorianDataSource gains EventsEntered flag (set before ThrowOnRead check) (#1)
- Events_non_source_node test gains clarifying doc comment explaining the SDK base rejects
variable nodes (EventNotifier=None) for event reads before our override runs; the
override's source-guard is exercised by the promoted-without-source test instead (#2)
Joseph Doherty