Commit Graph

786 Commits

Author SHA1 Message Date
Joseph Doherty db6446ce73 Merge fix/670-opcuaclient-session-loss: OpcUaClient upstream session loss goes Bad and recovers (#670) 2026-10-11 08:41:48 -04:00
Joseph Doherty b028322568 fix(opcuaclient): #670 review — subscription loss with a live session, stale recovery latch
M1: a subscription that dies while the session lives (outage longer than
the subscription lifetime but shorter than the session timeout) used to
end Healthy with fresh keep-alive stamps and frozen values. PublishError
BadNoSubscription / BadSubscriptionIdInvalid naming a tracked
subscription, or a tracked subscription whose SDK PublishingStopped
persists one more keep-alive window + 5 s across good keep-alives, now
declares it lost: Degraded, outage published once, and nothing advances
LastSuccessfulRead until the session is released, so the #527 watchdog
re-initialises and re-subscribes. Both detectors wire-tested.

M2: in-place recovery is bound to the session whose keep-alive recovered;
one overtaken by a re-initialise no longer re-arms the new session's
outage latch or degrades its health (ReadCoreAsync expectedSession).

L1 CAS health updates; L2 re-check before each recovery publish; L3 skip
CloseSession on a lost session; L4 SDK log forwarding rate-limited
(20/min); L5 per-driver SDK LoggerFactory disposed with the driver.
Wire test asserts the read stamp stays frozen while the server is down.
2026-10-11 08:41:27 -04:00
Joseph Doherty d803e0eab4 Merge fix/669-event-continuation: event HistoryRead continuation points + open-ended reads (#669) 2026-10-11 08:37:39 -04:00
Joseph Doherty 50b504d293 fix(opcua): raw open-ended/end-only reads per Part 11; extreme-bound conformance (#669) 2026-10-11 08:37:24 -04:00
Joseph Doherty b3db64af6c Merge feat/667-sqlserver-retention: SQL Server historian retention/purge (#667) 2026-10-11 08:27:47 -04:00
Joseph Doherty e4123ada91 fix(historian): retention applock release hardening, shutdown-quiet cancellation, release tests, docs (#667 review) 2026-10-11 08:27:31 -04:00
Joseph Doherty 2a8f9af5e4 fix(opcuaclient): upstream session loss goes Bad and recovers (#670)
A bad keep-alive now degrades health (LastSuccessfulRead preserved) and,
once per outage, publishes BadNoCommunication (value retained) for every
subscribed ref plus a quality-only annotation on subscribed alarm
conditions. Good keep-alives and data/event notifications advance
LastSuccessfulRead, so an idle-but-alive subscription never looks dead
to the #527 watchdog. A keep-alive that recovers on the same session
re-reads the Bad refs and republishes them Good; a session that does not
come back is recovered by the watchdog's ShutdownAsync + InitializeAsync
+ re-subscribe.

Deletes the SessionReconnectHandler wiring (it retried forever in SDK
1.5.378, cloned subscriptions the driver never re-tracked and kept health
Healthy), OnReconnectComplete, TryRearmReconnect and the reconnect_*
counters (new keepalive_recovered). Teardown of a lost session skips
wire deletes and bounds CloseSession. SDK warnings reach the driver log.
Wire test reworked: restart => Bad, then watchdog-style reinit => Good.
2026-10-11 08:27:04 -04:00
Joseph Doherty 924eff8d3c feat(historian): SQL Server retention purge (RetentionDays, Primary-gated, applock) (#667) 2026-10-11 08:21:30 -04:00
Joseph Doherty e5d12d391a fix(focas): publish LastStatus after the fixed-tree stamp; test waits on full readiness (#673) 2026-10-11 08:16:00 -04:00
Joseph Doherty 8d82d049e9 fix(historian): QuestDB final-review fix wave (I-1, M-1, M-3, M-5, M-8, M-9)
- I-1: QuestDbConnection caps Npgsql Maximum Pool Size at 24 (DefaultMaxPoolSize)
  when the connection string does not set it; an explicit value is kept. The
  effective value is logged once at Information.
- M-1: QuestDbTagCatalog.EnsureAsync waits for the tag table's sequenced commits
  and re-checks before inserting, so it no longer overwrites a provisioner row
  (unit/description) that is committed but not yet applied. Integration test
  reproduces the clobber against real QuestDB (falsified).
- M-3: the pre-1970 drop Warning is rate-limited per tag (once per 10 min);
  DroppedPre1970 still counts every sample.
- M-5: QuestDbHistorianProvider is IAsyncDisposable/IDisposable and disposes the
  shared Npgsql pool; Program.cs registers the provider through a factory and
  every seam resolves it from the container, so DI disposes it at shutdown.
- M-8, M-9: docs fixes (HistoryUpdate transaction wording; folded duplicate
  HistoryUpdate paragraph). Pool-size docs updated for the new default cap.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:35:35 -04:00
Joseph Doherty f661c084d8 test(historian): QuestDB-backed paging regression (T7 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:03:13 -04:00
Joseph Doherty 0e2bffc74a feat(historian): QuestDb provider + Host wiring; conformance suite green on both backends
AtTime conformance stamps are the requested time (M3; identical on SQL Server). The
QuestDB test fixture prunes idle pooled connections after 1 s: undisposed per-test data
sources exhausted QuestDB's 64-connection PG-wire limit in a full run.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:53:40 -04:00
Joseph Doherty 3244f9781a refactor(historian): one QuestDbTimeClusters helper for writer + updater existence reads; redelivery tombstone test
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:32:11 -04:00
Joseph Doherty d2be213e86 fix(historian): page by the backend's timestamp resolution; QuestDB AtTime stamps the requested time
IHistorianDataSource.TimestampResolutionTicks (default 1; QuestDB 10). A drained tie
cluster now steps off T by the resolution, so a forward one-value-per-page read on a
microsecond backend no longer re-reads the same sample forever (C1). QuestDB AtTime
results carry the client's original request time (M3); regression for the NEXT seek
skipping a tombstone (M1). Conformance backend wires the real writer and updater.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:30:25 -04:00
Joseph Doherty 13d628a07f feat(historian): QuestDb provider, registry + consumer-gated validator, compose env
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:23:21 -04:00
Joseph Doherty 18e85265f9 Merge qdb/T5: QuestDB reads and aggregates
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:19:32 -04:00
Joseph Doherty 3790a21acb Merge qdb/T4: QuestDB writes (value writer, provisioner, alarm writer)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:49 -04:00
Joseph Doherty 9d90dc9457 fix(historian): clustered existence reads + tag-predicate tests (T4 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:14 -04:00
Joseph Doherty c4b4b2e2bc feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)
QuestDbHistorianDataSource serves Raw (LIMIT $n, no continuation, R8), AtTime
(chunked LIMIT 1 neighbour seeks, <=50 times per statement), Events and
Processed (one statement per node, bucket = (cast(ts AS LONG)*10 - start)/interval,
shared BucketPlan/AggregateEvaluator/AtTimeEvaluator). Every sample read filters
NOT deleted; every row goes through StoredSample.FromStored; RC2 long.MinValue
read repair. QuestDbConformanceBackend (seed/read-back) for the conformance suite;
writer/updater factories are T7 placeholders.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:13:00 -04:00
Joseph Doherty 5c0671e5b5 fix(historian): per-tag update lock + clustered existence reads (T6 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:12:09 -04:00
Joseph Doherty 4c053cc4d0 feat(historian): QuestDB live writes, provisioning and alarm events
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:08:04 -04:00
Joseph Doherty ee174c0179 feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:04:48 -04:00
Joseph Doherty ecc0f914cc Merge qdb/T3: QuestDB historian skeleton
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:55:28 -04:00
Joseph Doherty 11b703708c feat(historian): QuestDB backend skeleton — options, connection, schema bootstrap with TTL
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:52:57 -04:00
Joseph Doherty c5b33aa123 test(historian): backend conformance suite (shared source), run against SQL Server
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:50:51 -04:00
Joseph Doherty 082e808d0d test(historian): pin Uncertain-input degradation (T1 review)
Two Sql.Common tests close review finding I-1: an Uncertain NEXT neighbour makes an AtTime
interpolation Uncertain, and an Uncertain (non-Bad) sample in a bucket makes the aggregate
Uncertain. Each falsified with the reviewer's surviving mutation.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:42:22 -04:00
Joseph Doherty f0700571e8 test(historian): direct Sql.Common evaluator tests (1b)
51 new test cases (34 methods) in Sql.Common.Tests covering AtTimeEvaluator, AggregateEvaluator
(hand-built tag-A bucket rows, ProcessedRead literals), BucketPlan, LiveWritePreparer and
HistoryUpdatePlanner (incl. resolutionTicks=10 truncate-before-dedup and the superseded-duplicate
outcome), plus EventContinuation, HistorianHealthCounters, ClampSeverity and StoredSample.FromStored.
Each was falsified by reverting the rule it pins (37 mutations, all red, all restored).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:35:51 -04:00
Joseph Doherty 8c6d6aff6a refactor(historian): extract Driver.Historian.Sql.Common from the SQL Server backend (no behaviour change) (1a)
Moves the database-independent logic of the SQL Server historian backend into a new
shared library for the upcoming QuestDB backend (#666): status codes and bits, value/column
mapping, StoredSample + quality clamp, the AtTime and aggregate evaluators, BucketPlan,
LiveWritePreparer and HistoryUpdatePlanner (both with a resolutionTicks parameter, default 1),
tag-name rules, the bootstrap failure cache, health counters, alarm event ids, the event
continuation and severity clamp. The SQL Server backend now calls it; its T-SQL, TVP
builders, connection, bootstrapper and catalog stay.

Tests: StatusCodeOracleTests, ValueWriterPrepareTests, the column half of SampleRowMapperTests
and ProcessedReadTests.Every_enum_member_is_either_served_or_refused move to the new
Sql.Common.Tests under the same names. SqlServer integration tests get mechanical renames
only. Test-name set across the three projects is identical to the baseline (160 tests).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:26:49 -04:00
Joseph Doherty 60a1e7e669 docs(historian): fix-wave docs and stale comments
- Historian.md: Empty samples always Bad (writer/updater/read clamp),
  long-name refusal, 5 s bootstrap failure cache, AtTime after now, info-bit
  masking, HistoryUpdate does NOT type-check (M-1 decision), DeleteRaw
  needs both bounds, reverse Raw paging, the historian/historization metric
  prefixes, and provisioning numeric-only lines qualified as Gateway.
- CLAUDE.md numeric-only lines qualified as Gateway backend.
- Stale 'sole historian' lines in DriverLifecycle, ServiceHosting and the
  retired Wonderware stub; Client.CLI exit-code contract.
- ServerHistorianOptions: Endpoint/Backend XML docs untangled, Validate()
  indented; Program.cs comments are backend-neutral; T10 week-test day count.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:33:30 -04:00
Joseph Doherty c6d4134795 test(historian): gateway writer acks an all-non-numeric batch without a call; fix stale writer docs
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:24:45 -04:00
Joseph Doherty 821763a09d fix(historian): SQL quality contract, long-name refusal, info-bit masking, AtTime-after-now, bootstrap hardening
- I-1: an Empty sample is always Bad. The writer stores a non-Bad NaN/Inf as
  Empty+BadOutOfRange and any other non-Bad Empty as BadNoData; the updater
  refuses a non-Bad Empty with BadInvalidArgument; reads clamp a legacy
  non-Bad Empty row to BadNoData (Raw and AtTime).
- Tag/source names over 400 chars are refused, never truncated: writer
  per-tag failure, updater BadInvalidArgument, reads not-found, events none.
- HistorianInfoBits masks the historian bits before setting one.
- AtTime past now, stepped from the last sample => UncertainDataSubNormal.
- Bootstrapper: logger, 5 s negative cache, ClearPool + Warning when
  sp_releaseapplock fails.
- Provider builds its shared state lazily (an invalid Schema no longer
  throws at Resolve); Kinds is a FrozenSet; Gateway reuses the recorder's
  DefaultSupportedKinds.
- Cleanup: SqlHistorianDataTypes in its own file (updater dup removed),
  dead Invalidate and unused status consts deleted, named severity masks,
  alarm writer brace.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:16:31 -04:00
Joseph Doherty 7500106c2f feat(historian): SqlServerHistorianProvider + SqlServer backend registration
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:42:38 -04:00
Joseph Doherty 0a8deba94c Merge ph-T11 (pluggable historian)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:34:53 -04:00
Joseph Doherty a8c0919365 Merge ph-T10 (pluggable historian)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:34:20 -04:00
Joseph Doherty 8f859acb2c fix(historian): T10 review — tick-exact bucketing over the full DateTime range; AtTime outage keeps its Bad code
Important 1: bucket offsets are now 100 ns ticks as bigint
(DATEDIFF_BIG(microsecond)*10 plus the sub-microsecond tick digits), so a
1601→9999 window no longer overflows (SQL 535) and nothing scales the interval
to nanoseconds in C#. Interval wider than the window clamps to one bucket and
the ceiling division cannot overflow. Exactness to the tick is unchanged.

Important 2 (spec §4 ruling d6afb394): a Bad previous neighbour means the time
is inside an outage — null value with that neighbour's Bad code, never
Uncertain. Usable previous + Bad next still holds the previous value Uncertain.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:30:54 -04:00
Joseph Doherty c2ab68ceb5 fix(historian): T9 review round 1 - dup-key retry counter, alarm Classify retries unknown, test hardening
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:30:42 -04:00
Joseph Doherty 38d37f5c23 feat(historian): SQL Server read side — raw, at-time, events, push-down processed (ph-T10)
SqlServerHistorianDataSource implements IHistorianDataSource over the T8 schema:
raw forward/reverse/point with the 0 = unbounded sentinel; AtTime in one statement
(TimeRow TVP + two OUTER APPLYs) with linear/stepped/no-prior rules and no
interpolation across a Bad neighbour or Empty outage marker; events with a
maxEvents+1 truncation probe and a non-null continuation point; processed
aggregates pushed down to T-SQL (SqlAggregateQueries) with nanosecond-exact
bucket edges, Calculated bits on every result, Uncertain on any non-Good input,
BadAggregateInvalidInputs for numeric aggregates over String/DateTime, a
100 000-bucket cap and HistoryAggregateNotSupportedException for the rest.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:13:07 -04:00
Joseph Doherty bef47f89da feat(historian): SQL Server value writer, tag provisioner, alarm writer (T9)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:08:43 -04:00
Joseph Doherty 2d4fbc857f feat(historian): SQL Server HistoryUpdate data updater (T11)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:07:44 -04:00
Joseph Doherty 65e6352dad Merge ph-T8 (pluggable historian)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:00:51 -04:00
Joseph Doherty 4fa6cf8fee Merge ph-T4 (pluggable historian)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:57:52 -04:00
Joseph Doherty 17962d4233 feat(historian): SQL Server backend skeleton - schema DDL, locked bootstrapper, options, connection/mapper/tag-catalog helpers, status consts, env-gated fixture
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:52:15 -04:00
Joseph Doherty 29893ef1b4 feat(historian): outbox format v2 carries typed values + status; legacy entries still read
Serialize always writes v2 ("HV2\0" | ver 2 | id | tag | kind | payload | status u32 | ts).
TryDeserialize reads v2 by magic, falls back to the legacy exact-length layout (also when the
magic matches but v2 validation fails), else returns false - never throws.

FasterLogHistorizationOutbox: RecoverState and PeekBatchAsync both use TryDeserialize. An
unreadable record is indexed under a synthetic id (FIFO truncation passes it), skipped by peek,
counted once (UnreadableCount), and dropped by peek once it reaches the head so it can never
pin the outbox.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:50:25 -04:00
Joseph Doherty 8c8e243de6 feat(historian): GatewayHistorianProvider; status→DA quality reverse mapping
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:47:09 -04:00
Joseph Doherty ff7c6c531a feat(historian): typed HistorianValue contracts, IHistorianDataUpdater, IHistorianBackendProvider (no behaviour change)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 12:44:19 -04:00
Joseph Doherty 1ea1da8236 refactor(runtime): delete the dead in-place ApplyDelta seam; config change = respawn only (#665)
Since #516 a DriverConfig change is stop + respawn, so DriverSpawnPlan.ToApplyDelta
was always empty and the in-place path had no producer. #663 and #664 were both
latent bugs in it. This deletes it:
- host: ApplyChildDelta, _pendingDelta, HandleApplyResult and HandleDeltaApplied
- plan: DriverSpawnPlan.ToApplyDelta
- actor: the ApplyDelta, ApplyResult and DeltaApplied messages, the Connected
  reinit handler and the mid-connect AdoptConfigDuringInit
- tests: the actor tests that drove the seam

IDriver.ReinitializeAsync and each driver's implementation and unit tests stay.
Comments and docs that cited AdoptConfigDuringInit as the reason a driver must
release prior state at init now cite the IDriver contract and overlapping
retries, which still apply.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 11:23:16 -04:00
Joseph Doherty 97fcf624ae fix(drivers): mid-connect ApplyDelta adopts config in Calculation, refuses it in Galaxy (#664)
AdoptConfigDuringInit re-runs InitializeAsync(newConfig) and acks success, but
Calculation and Galaxy ignored the JSON there: Calculation kept its old tag
set, and Galaxy connected on its old endpoint. Calculation now rebuilds its tag
table from a config with a body (sharing AdoptConfig with ReinitializeAsync and
stopping any running timer scheduler first). Galaxy, which cannot hot-swap its
options, refuses a non-equivalent one the way ReinitializeAsync does. The actor
now sends DeltaApplied once the adopted init succeeds, so a dependency
consumer's mux interest is re-registered. Unreachable from deploys since #516.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 11:05:58 -04:00
Joseph Doherty ea220363f3 Merge hw5-657 (#657)
v2-ci / build (push) Successful in 5m44s
v2-ci / unit-tests (push) Failing after 24m4s
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 07:15:30 -04:00
Joseph Doherty b394413e30 Merge hw5-592 (#592)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 07:15:30 -04:00
Joseph Doherty ede40b6c6d Merge hw5-660 (#660)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 07:15:30 -04:00