dohertj2/histsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M4 R4.1: pragmatic store-and-forward durable outbox

Browse Source 
Adds AVEVA.Historian.Client.StoreForward — a client-side store-and-forward
layer over the historian write surface (AddHistoricalValuesAsync /
SendEventAsync). Producers enqueue writes; the writer persists them and
replays on reconnect so a transient disconnect never drops data. This is the
roadmap's recommended pragmatic outbox, NOT a bit-faithful reimplementation of
AVEVA's native SF cache (that stays deferred) — pure managed, no RE.

- HistorianOutboxEntry / HistorianOutboxEntryKind: buffered-write envelope
- IHistorianOutboxStore + InMemoryHistorianOutboxStore (tests) +
  FileHistorianOutboxStore (crash-durable: atomic temp+move JSON per entry,
  FIFO by filename sequence that resumes past on-disk max, corrupt-file
  quarantine). OutboxJson normalizes event object? properties off JsonElement.
- IHistorianWriteSink + HistorianClientWriteSink (HistorianClient-backed)
- HistorianStoreForwardWriter: enqueue, single-flight FIFO FlushAsync with
  head-of-line blocking, optional MaxDeliveryAttempts dead-lettering,
  DropOldest/Reject overflow policy, background drain loop (retry on reconnect),
  GetStatusAsync snapshot mirroring server SF Pending/Storing/ErrorOccurred.

12 unit tests (no server): durability-across-restart, reconnect-drain, FIFO
order/head-of-line, dead-letter, overflow policies, background auto-drain.
Full suite 293 green. Roadmap R4.1 marked shipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 22:35:30 -04:00
parent a91f126287
commit dd2aec3b8b
14 changed files with 1352 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/hcal-roadmap.md
+2 -2
View File
@@ -271,7 +271,7 @@ Only if the use case demands them. Each is a real subsystem, not an op.
| ID | Capability | Approach | Risk |
|---|---|---|---|
| R4.1 | Store-and-forward | **Pragmatic local queue** (durable outbox + replay on reconnect) rather than bit-faithful SF cache + `Forward*Snapshot`. Faithful SF = decode SF cache format + snapshot framing + recovery log | high; consider "good enough" |
| R4.1 | Store-and-forward | ✅ **SHIPPED (2026-06-21) — pragmatic durable outbox.** `AVEVA.Historian.Client.StoreForward`: `HistorianStoreForwardWriter` buffers historical-value + event writes to an `IHistorianOutboxStore` (`FileHistorianOutboxStore` = crash-durable atomic JSON-per-entry, FIFO by filename sequence, corrupt-file quarantine; `InMemoryHistorianOutboxStore` for tests) and replays them through an `IHistorianWriteSink` (default `HistorianClientWriteSink`). Background drain loop retries on reconnect; FIFO head-of-line blocking with optional `MaxDeliveryAttempts` dead-lettering; `DropOldest`/`Reject` overflow policy; `GetStatusAsync` snapshot (Pending/Storing/ErrorOccurred mirrors the server SF semantics). 12 unit tests (durability-across-restart, reconnect-drain, head-of-line order, dead-letter, overflow, background loop). **NOT** the bit-faithful native SF cache (`Forward*Snapshot` decode) — that stays deferred; pure client-side, no RE. | high; consider "good enough" |
| R4.2 | Revision / edit writes | `AddRevisionValue(s)` go via the **non-WCF storage-engine pipe** (`STransactPipeClient2`) — separate transport RE | high |
| R4.3 | Real store-forward **status** | duplex push (`SetStoreForwardEvent`) or a decoded pull endpoint — see store-forward plan | medium |
| R4.4 | Multi-historian / redundancy | client-side orchestration over N single-historian sessions (failover, ReSyncTags, partner watchdog) — build last | medium |
@@ -331,4 +331,4 @@ event-send). M3/M4 as demand dictates.
| M1 cheap surface | TRIVIAL/BOUNDED | M–L | most remaining read/config | ✅ **done** (reachable surface; rest bounded out) |
| M2 event send | CAPTURE | S–M | headline write capability | ✅ **done** |
| M3 historical writes | BOUNDED | M | backfill | ✅ **SHIPPED + LIVE-VALIDATED (2026-06-21)** — `AddHistoricalValuesAsync` over gRPC = `HistoryService.AddStreamValues` ("ON" buffer) + tag-GUID resolve. Pure-managed SDK write read back live. All 5 analog types (Float/Double/Int2/Int4/UInt4). WCF still blocked (D2) |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | defer (R4.2 = same pipe wall) |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | **R4.1 store-and-forward SHIPPED** (pragmatic durable outbox, 2026-06-21); R4.2 revisions deferred (same pipe wall), R4.3 SF status + R4.4 redundancy deferred |