Commit Graph

4016 Commits

Author SHA1 Message Date
Joseph Doherty 2a8f9af5e4 fix(opcuaclient): upstream session loss goes Bad and recovers (#670)
A bad keep-alive now degrades health (LastSuccessfulRead preserved) and,
once per outage, publishes BadNoCommunication (value retained) for every
subscribed ref plus a quality-only annotation on subscribed alarm
conditions. Good keep-alives and data/event notifications advance
LastSuccessfulRead, so an idle-but-alive subscription never looks dead
to the #527 watchdog. A keep-alive that recovers on the same session
re-reads the Bad refs and republishes them Good; a session that does not
come back is recovered by the watchdog's ShutdownAsync + InitializeAsync
+ re-subscribe.

Deletes the SessionReconnectHandler wiring (it retried forever in SDK
1.5.378, cloned subscriptions the driver never re-tracked and kept health
Healthy), OnReconnectComplete, TryRearmReconnect and the reconnect_*
counters (new keepalive_recovered). Teardown of a lost session skips
wire deletes and bounds CloseSession. SDK warnings reach the driver log.
Wire test reworked: restart => Bad, then watchdog-style reinit => Good.
2026-10-11 08:27:04 -04:00
Joseph Doherty db6a53036b docs(plans): issue batch #667-#673 design 2026-10-11 08:14:38 -04:00
Joseph Doherty ee67b6370a Merge feat/questdb-historian: QuestDB historian backend (#666)
v2-ci / build (push) Successful in 5m6s
v2-ci / unit-tests (push) Failing after 23m29s
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 04:40:42 -04:00
Joseph Doherty 1569a59ad2 test(historian): QuestDB live gate PASSED on docker-dev (6/6 legs); RetentionDays compose key
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 04:00:03 -04:00
Joseph Doherty 2ebe0a4226 Merge qdb/final: final-review fix wave (pool cap, catalog metadata, warn rate-limit, disposal, docs)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:37:55 -04:00
Joseph Doherty 8d82d049e9 fix(historian): QuestDB final-review fix wave (I-1, M-1, M-3, M-5, M-8, M-9)
- I-1: QuestDbConnection caps Npgsql Maximum Pool Size at 24 (DefaultMaxPoolSize)
  when the connection string does not set it; an explicit value is kept. The
  effective value is logged once at Information.
- M-1: QuestDbTagCatalog.EnsureAsync waits for the tag table's sequenced commits
  and re-checks before inserting, so it no longer overwrites a provisioner row
  (unit/description) that is committed but not yet applied. Integration test
  reproduces the clobber against real QuestDB (falsified).
- M-3: the pre-1970 drop Warning is rate-limited per tag (once per 10 min);
  DroppedPre1970 still counts every sample.
- M-5: QuestDbHistorianProvider is IAsyncDisposable/IDisposable and disposes the
  shared Npgsql pool; Program.cs registers the provider through a factory and
  every seam resolves it from the container, so DI disposes it at shutdown.
- M-8, M-9: docs fixes (HistoryUpdate transaction wording; folded duplicate
  HistoryUpdate paragraph). Pool-size docs updated for the new default cap.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:35:35 -04:00
Joseph Doherty e177a50046 Merge qdb/T8: QuestDB backend documentation (as built)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:10:56 -04:00
Joseph Doherty 94691ec50f docs(historian): QuestDB backend documentation (as built)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:09:11 -04:00
Joseph Doherty 17cff2ba16 Merge qdb/T7: QuestDb provider + Host wiring; conformance green on both backends
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:04:43 -04:00
Joseph Doherty f661c084d8 test(historian): QuestDB-backed paging regression (T7 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:03:13 -04:00
Joseph Doherty 0e2bffc74a feat(historian): QuestDb provider + Host wiring; conformance suite green on both backends
AtTime conformance stamps are the requested time (M3; identical on SQL Server). The
QuestDB test fixture prunes idle pooled connections after 1 s: undisposed per-test data
sources exhausted QuestDB's 64-connection PG-wire limit in a full run.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:53:40 -04:00
Joseph Doherty 3244f9781a refactor(historian): one QuestDbTimeClusters helper for writer + updater existence reads; redelivery tombstone test
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:32:11 -04:00
Joseph Doherty d2be213e86 fix(historian): page by the backend's timestamp resolution; QuestDB AtTime stamps the requested time
IHistorianDataSource.TimestampResolutionTicks (default 1; QuestDB 10). A drained tie
cluster now steps off T by the resolution, so a forward one-value-per-page read on a
microsecond backend no longer re-reads the same sample forever (C1). QuestDB AtTime
results carry the client's original request time (M3); regression for the NEXT seek
skipping a tombstone (M1). Conformance backend wires the real writer and updater.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:30:25 -04:00
Joseph Doherty 13d628a07f feat(historian): QuestDb provider, registry + consumer-gated validator, compose env
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:23:21 -04:00
Joseph Doherty 18e85265f9 Merge qdb/T5: QuestDB reads and aggregates
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:19:32 -04:00
Joseph Doherty 3790a21acb Merge qdb/T4: QuestDB writes (value writer, provisioner, alarm writer)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:49 -04:00
Joseph Doherty 9d90dc9457 fix(historian): clustered existence reads + tag-predicate tests (T4 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:14 -04:00
Joseph Doherty e5b9783ba3 Merge qdb/T6: QuestDB HistoryUpdate (soft delete)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:13:25 -04:00
Joseph Doherty c4b4b2e2bc feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)
QuestDbHistorianDataSource serves Raw (LIMIT $n, no continuation, R8), AtTime
(chunked LIMIT 1 neighbour seeks, <=50 times per statement), Events and
Processed (one statement per node, bucket = (cast(ts AS LONG)*10 - start)/interval,
shared BucketPlan/AggregateEvaluator/AtTimeEvaluator). Every sample read filters
NOT deleted; every row goes through StoredSample.FromStored; RC2 long.MinValue
read repair. QuestDbConformanceBackend (seed/read-back) for the conformance suite;
writer/updater factories are T7 placeholders.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:13:00 -04:00
Joseph Doherty 5c0671e5b5 fix(historian): per-tag update lock + clustered existence reads (T6 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:12:09 -04:00
Joseph Doherty 4c053cc4d0 feat(historian): QuestDB live writes, provisioning and alarm events
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:08:04 -04:00
Joseph Doherty ee174c0179 feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:04:48 -04:00
Joseph Doherty ecc0f914cc Merge qdb/T3: QuestDB historian skeleton
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:55:28 -04:00
Joseph Doherty fabcc6eed6 Merge qdb/T2: historian conformance suite over SQL Server
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:52:59 -04:00
Joseph Doherty 11b703708c feat(historian): QuestDB backend skeleton — options, connection, schema bootstrap with TTL
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:52:57 -04:00
Joseph Doherty c5b33aa123 test(historian): backend conformance suite (shared source), run against SQL Server
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:50:51 -04:00
Joseph Doherty ca2c20f852 Merge qdb/T1: extract Driver.Historian.Sql.Common
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:43:01 -04:00
Joseph Doherty 082e808d0d test(historian): pin Uncertain-input degradation (T1 review)
Two Sql.Common tests close review finding I-1: an Uncertain NEXT neighbour makes an AtTime
interpolation Uncertain, and an Uncertain (non-Bad) sample in a bucket makes the aggregate
Uncertain. Each falsified with the reviewer's surviving mutation.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:42:22 -04:00
Joseph Doherty f0700571e8 test(historian): direct Sql.Common evaluator tests (1b)
51 new test cases (34 methods) in Sql.Common.Tests covering AtTimeEvaluator, AggregateEvaluator
(hand-built tag-A bucket rows, ProcessedRead literals), BucketPlan, LiveWritePreparer and
HistoryUpdatePlanner (incl. resolutionTicks=10 truncate-before-dedup and the superseded-duplicate
outcome), plus EventContinuation, HistorianHealthCounters, ClampSeverity and StoredSample.FromStored.
Each was falsified by reverting the rule it pins (37 mutations, all red, all restored).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:35:51 -04:00
Joseph Doherty 8c6d6aff6a refactor(historian): extract Driver.Historian.Sql.Common from the SQL Server backend (no behaviour change) (1a)
Moves the database-independent logic of the SQL Server historian backend into a new
shared library for the upcoming QuestDB backend (#666): status codes and bits, value/column
mapping, StoredSample + quality clamp, the AtTime and aggregate evaluators, BucketPlan,
LiveWritePreparer and HistoryUpdatePlanner (both with a resolutionTicks parameter, default 1),
tag-name rules, the bootstrap failure cache, health counters, alarm event ids, the event
continuation and severity clamp. The SQL Server backend now calls it; its T-SQL, TVP
builders, connection, bootstrapper and catalog stay.

Tests: StatusCodeOracleTests, ValueWriterPrepareTests, the column half of SampleRowMapperTests
and ProcessedReadTests.Every_enum_member_is_either_served_or_refused move to the new
Sql.Common.Tests under the same names. SqlServer integration tests get mechanical renames
only. Test-name set across the three projects is identical to the baseline (160 tests).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:26:49 -04:00
Joseph Doherty 27991434a0 spike(historian): T0 review fixes
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:10:31 -04:00
Joseph Doherty cdea173bd1 spike(historian): QuestDB behaviours confirmed against pinned image; spec §2–§4 amended
questdb/questdb:10.0.1 (pushed 2026-08-24). Q1–Q13 answered with harness output in
docs/plans/2026-10-10-questdb-spike.md; confirmed DDL, visibility wait, bucket/neighbour
SQL and connection keywords folded into the design spec, marked T0-confirmed.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:07:38 -04:00
Joseph Doherty f54ab04e64 docs(historian): fold Fable pre-execution review into QuestDB plan + spec (6 blockers, 15 should-fix, 8 nits)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-10 09:09:42 -04:00
Joseph Doherty b3031dfea9 docs(historian): QuestDB backend implementation plan (#666)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-10 08:56:15 -04:00
Joseph Doherty 9998d816f1 docs(historian): QuestDB historian backend design (#666)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-10 08:46:48 -04:00
Joseph Doherty 8ec7a83c83 Merge feat/pluggable-historian: pluggable historian backends (Gateway | SqlServer), typed values + real status, OPC UA HistoryUpdate
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 15:29:10 -04:00
Joseph Doherty 5fdf205caa test(historian): live gate PASSED on docker-dev (SqlServer backend); CLI prints node-level HistoryUpdate result
Gate: Primary-only typed recording with real status, raw+UNS HistoryRead/Processed parity, HistoryUpdate
insert/duplicate/replace/delete + WriteOperate-only and anonymous denied, failover at 1x, Modbus device loss
stored as BadCommunicationError then a BadNoCommunication outage marker. Gate found the CLI hid the
node-level refusal code (fixed + pinned) and a pre-existing OpcUaClient comms-loss gap (filed separately).
docker-dev compose gains ServerHistorian__Backend / __SqlServer__ConnectionString (default Gateway).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 15:14:46 -04:00
Joseph Doherty e2011a653e Merge ph-fix (pluggable historian final fix wave)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:40:45 -04:00
Joseph Doherty 159d18de59 docs(plans): record the fix-wave rulings in the design's as-built notes
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:33:45 -04:00
Joseph Doherty 60a1e7e669 docs(historian): fix-wave docs and stale comments
- Historian.md: Empty samples always Bad (writer/updater/read clamp),
  long-name refusal, 5 s bootstrap failure cache, AtTime after now, info-bit
  masking, HistoryUpdate does NOT type-check (M-1 decision), DeleteRaw
  needs both bounds, reverse Raw paging, the historian/historization metric
  prefixes, and provisioning numeric-only lines qualified as Gateway.
- CLAUDE.md numeric-only lines qualified as Gateway backend.
- Stale 'sole historian' lines in DriverLifecycle, ServiceHosting and the
  retired Wonderware stub; Client.CLI exit-code contract.
- ServerHistorianOptions: Endpoint/Backend XML docs untangled, Validate()
  indented; Program.cs comments are backend-neutral; T10 week-test day count.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:33:30 -04:00
Joseph Doherty 7fafeb433c test(opcua): flag-OFF HistoryUpdate over the wire - Cluster HistoryUpdate grant allowed, WriteOperate alone denied
The rig's key roles are now settable; the test derives them with the
production ApiKeyInterimRoleMapper from the same grants it installs.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:31:01 -04:00
Joseph Doherty f8ad74dc19 fix(opcua): DeleteRawModified with an unspecified bound is BadInvalidArgument; HistoryUpdate node-manager tests
- A DateTime.MinValue start or end on DeleteRawModified is refused
  BadInvalidArgument before the backend.
- Tests: anonymous without the HistoryUpdate role denied (with it allowed),
  UpdateTagAttributes enabling historization sets HistoryWrite, a throwing
  updater is BadInternalError, the MinValue bounds case; corrected the
  test helper's comment on how the SDK base seeds unadmitted slots.
- GatewayHistorianValueWriter remarks describe the recorder's real status
  capture (and that it does not persist on the gateway).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:27:33 -04:00
Joseph Doherty c6d4134795 test(historian): gateway writer acks an all-non-numeric batch without a call; fix stale writer docs
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:24:45 -04:00
Joseph Doherty 96ec751a0b fix(historian): latch the recorder's dropped-kind Debug log per run; pin the outbox.unreadable gauge
The DroppedUnsupportedKind counter still counts every value; only the Debug
line is latched (cleared by the next captured value), so a Gateway-backend
comms loss no longer logs one line per tag.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:23:42 -04:00
Joseph Doherty 7c787b8b64 test(host): an invalid SqlServer:Schema is consumer-gated, not a Resolve-time throw
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:22:41 -04:00
Joseph Doherty 6e10e45bb9 fix(cli): historyupdate exits non-zero unless every item is Good, and never prints a blank status
A Good service result with missing or too few per-item results, a missing
result entry, or an Uncertain item now fails the command; a status code
the SDK has no name for prints as hex.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:21:32 -04:00
Joseph Doherty 8d51380ff1 fix(opcua): reverse HistoryRead-Raw paging no longer stops after the first page
A reverse read (start > end) with a small NumValuesPerNode resumed through
the tie-cluster path, which drained the cluster and stepped FORWARD to
T + 1 tick - outside a reverse window - so the read ended with GoodNoData
after one sample. SliceTieCluster now steps down to T - 1 tick for a
reverse read (terminal at the excluded end), and the node manager slices
the over-fetched cluster in the read's own direction so the skip count is
not applied from the wrong end. Forward paging is unchanged.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:20:00 -04:00
Joseph Doherty 821763a09d fix(historian): SQL quality contract, long-name refusal, info-bit masking, AtTime-after-now, bootstrap hardening
- I-1: an Empty sample is always Bad. The writer stores a non-Bad NaN/Inf as
  Empty+BadOutOfRange and any other non-Bad Empty as BadNoData; the updater
  refuses a non-Bad Empty with BadInvalidArgument; reads clamp a legacy
  non-Bad Empty row to BadNoData (Raw and AtTime).
- Tag/source names over 400 chars are refused, never truncated: writer
  per-tag failure, updater BadInvalidArgument, reads not-found, events none.
- HistorianInfoBits masks the historian bits before setting one.
- AtTime past now, stepped from the last sample => UncertainDataSubNormal.
- Bootstrapper: logger, 5 s negative cache, ClearPool + Warning when
  sp_releaseapplock fails.
- Provider builds its shared state lazily (an invalid Schema no longer
  throws at Resolve); Kinds is a FrozenSet; Gateway reuses the recorder's
  DefaultSupportedKinds.
- Cleanup: SqlHistorianDataTypes in its own file (updater dup removed),
  dead Invalidate and unused status consts deleted, named severity masks,
  alarm writer brace.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 14:16:31 -04:00
Joseph Doherty fc57eefca6 Merge ph-T14 (pluggable historian docs)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:52:14 -04:00
Joseph Doherty cc34daf181 docs(historian): pluggable backends, SQL Server backend, HistoryUpdate, outbox v2, spec corrections
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-09 13:51:56 -04:00