Commit Graph

14 Commits

Author SHA1 Message Date
Joseph Doherty cfd403710e feat(historian): Part 11 modified-value history on SqlServer + QuestDB (#668)
Every HistoryUpdate a SQL backend applies now leaves a modification trail
(prior value, principal, Part 11 update type, modification time), and
HistoryReadRaw with IsReadModified=true serves it as HistoryModifiedData.

- Core: HistoryUpdateContext threaded into all three IHistorianDataUpdater
  methods; IHistorianDataSource.ReadModifiedAsync (default NotSupported =>
  BadHistoryOperationUnsupported on Gateway/Null); HistoryModified* contracts.
- SqlServer: schema v2 (002_sample_modified.sql, RequiredVersion=2, docs copy);
  trail written by OUTPUT ... INTO on the same MERGE/DELETE statement, so a
  failed update leaves no audit row; retention purges SampleModified per tag on
  the source-time cutoff; ReadModifiedAsync keyed on (SourceTimeUtc, ModifiedId).
- QuestDB: append-only {prefix}sample_modified (TTL like samples), trail row
  written BEFORE the change (orphan-on-crash, never an untracked change);
  existence reads now carry the prior value; ReadModifiedAsync.
- Node manager: modified reads dispatch + page via the #669 continuation store
  (HistoryContinuationState gains HistoryCursorKind + LastModifiedId), gated by
  the existing HistoryRead decision; session principal / key id passed to the
  updater. IsDeleteModified stays unsupported.
- Client.CLI: historyread --modified.
- Tests: node-manager dispatch/paging/cursor binding/principal threading,
  over-the-wire HistoryModifiedData, shared ModifiedValueConformance on both
  backends, SqlServer atomicity (forced trail failure => no change) + v1->v2
  upgrade + retention trail purge.
- Docs: Historian.md (Modified-value history section, schema v2 upgrade note),
  security.md, Client.CLI.md, CLAUDE.md.
2026-10-11 09:26:36 -04:00
Joseph Doherty 8d82d049e9 fix(historian): QuestDB final-review fix wave (I-1, M-1, M-3, M-5, M-8, M-9)
- I-1: QuestDbConnection caps Npgsql Maximum Pool Size at 24 (DefaultMaxPoolSize)
  when the connection string does not set it; an explicit value is kept. The
  effective value is logged once at Information.
- M-1: QuestDbTagCatalog.EnsureAsync waits for the tag table's sequenced commits
  and re-checks before inserting, so it no longer overwrites a provisioner row
  (unit/description) that is committed but not yet applied. Integration test
  reproduces the clobber against real QuestDB (falsified).
- M-3: the pre-1970 drop Warning is rate-limited per tag (once per 10 min);
  DroppedPre1970 still counts every sample.
- M-5: QuestDbHistorianProvider is IAsyncDisposable/IDisposable and disposes the
  shared Npgsql pool; Program.cs registers the provider through a factory and
  every seam resolves it from the container, so DI disposes it at shutdown.
- M-8, M-9: docs fixes (HistoryUpdate transaction wording; folded duplicate
  HistoryUpdate paragraph). Pool-size docs updated for the new default cap.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:35:35 -04:00
Joseph Doherty f661c084d8 test(historian): QuestDB-backed paging regression (T7 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 03:03:13 -04:00
Joseph Doherty 0e2bffc74a feat(historian): QuestDb provider + Host wiring; conformance suite green on both backends
AtTime conformance stamps are the requested time (M3; identical on SQL Server). The
QuestDB test fixture prunes idle pooled connections after 1 s: undisposed per-test data
sources exhausted QuestDB's 64-connection PG-wire limit in a full run.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:53:40 -04:00
Joseph Doherty 3244f9781a refactor(historian): one QuestDbTimeClusters helper for writer + updater existence reads; redelivery tombstone test
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:32:11 -04:00
Joseph Doherty d2be213e86 fix(historian): page by the backend's timestamp resolution; QuestDB AtTime stamps the requested time
IHistorianDataSource.TimestampResolutionTicks (default 1; QuestDB 10). A drained tie
cluster now steps off T by the resolution, so a forward one-value-per-page read on a
microsecond backend no longer re-reads the same sample forever (C1). QuestDB AtTime
results carry the client's original request time (M3); regression for the NEXT seek
skipping a tombstone (M1). Conformance backend wires the real writer and updater.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:30:25 -04:00
Joseph Doherty 18e85265f9 Merge qdb/T5: QuestDB reads and aggregates
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:19:32 -04:00
Joseph Doherty 3790a21acb Merge qdb/T4: QuestDB writes (value writer, provisioner, alarm writer)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:49 -04:00
Joseph Doherty 9d90dc9457 fix(historian): clustered existence reads + tag-predicate tests (T4 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:16:14 -04:00
Joseph Doherty c4b4b2e2bc feat(historian): QuestDB reads and aggregates (pushed down, shared evaluation)
QuestDbHistorianDataSource serves Raw (LIMIT $n, no continuation, R8), AtTime
(chunked LIMIT 1 neighbour seeks, <=50 times per statement), Events and
Processed (one statement per node, bucket = (cast(ts AS LONG)*10 - start)/interval,
shared BucketPlan/AggregateEvaluator/AtTimeEvaluator). Every sample read filters
NOT deleted; every row goes through StoredSample.FromStored; RC2 long.MinValue
read repair. QuestDbConformanceBackend (seed/read-back) for the conformance suite;
writer/updater factories are T7 placeholders.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:13:00 -04:00
Joseph Doherty 5c0671e5b5 fix(historian): per-tag update lock + clustered existence reads (T6 review)
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:12:09 -04:00
Joseph Doherty 4c053cc4d0 feat(historian): QuestDB live writes, provisioning and alarm events
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:08:04 -04:00
Joseph Doherty ee174c0179 feat(historian): QuestDB HistoryUpdate with soft delete and read-your-writes
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 02:04:48 -04:00
Joseph Doherty 11b703708c feat(historian): QuestDB backend skeleton — options, connection, schema bootstrap with TTL
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
2026-10-11 01:52:57 -04:00