perf(historian): batch alarm-history writes; pipeline gateway SendEvent (lmxopcua#746)
All three alarm-history writers inserted one event per round trip, serially (SqlServer also opened a pooled connection per event), so a 10000-event alarm storm drained at 10000 x RTT while the backlog sat in LocalDb. - Sql.Common.AlarmBatchWrite: one statement per chunk; a chunk that fails PermanentFail-class is re-sent event by event so only the poison event dead-letters; a transient failure retries the chunk; a cancellation never dead-letters. - SqlServer: multi-row insert-if-absent over a VALUES set (150 events per statement, under the 2100-parameter limit), duplicate ids in a batch written once; a duplicate-key race isolates per event, where it is Ack. No new table type, so no schema migration (the review suggested a TVP; the VALUES form gets the same one-round-trip-per-chunk without one). - QuestDb: multi-row INSERTs of 500 events; pre-1970 events still dead-letter unsent. - Gateway: no batch RPC exists, so sends are pipelined, at most 8 in flight, outcomes kept in input order. Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22 in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
+6
-2
@@ -80,7 +80,7 @@ consumes the historian. The existing enable gates (`ServerHistorian:Enabled`, `A
|
||||
| Processed aggregates | Twelve: the ten per-sample ones (Average, Count, Minimum, Maximum, Range, Start, End, Delta, Min/MaxActualTime) evaluated from raw reads with the SQL backends' evaluator — identical results (#681) — plus TimeAverage and Total from AVEVA's native modes | The seventeen in the [aggregate table](#aggregates-processed-reads) (incl. Part 13 time-weighted, #671) | The same seventeen, identical results |
|
||||
| HistoryUpdate | No (`BadHistoryOperationUnsupported`) | Yes (Insert / Replace / Update / DeleteRaw / DeleteAtTime) | Yes — soft delete (tombstones), see [below](#historyupdate-soft-delete) |
|
||||
| Modified-value history (`IsReadModified`, #668) | No (`BadHistoryOperationUnsupported`) | Yes — `SampleModified`, atomic with the change | Yes — `{prefix}sample_modified`, written before the change (orphan-on-crash) |
|
||||
| Alarm history | gateway `SendEvent` | `Event` table (insert-if-absent) | `{prefix}event` table (dedup upsert on `(ts, event_id)`) |
|
||||
| Alarm history | gateway `SendEvent`, one RPC per event, up to 8 in flight (no batch RPC yet) | `Event` table (insert-if-absent), multi-row statements of 150 events | `{prefix}event` table (dedup upsert on `(ts, event_id)`), multi-row INSERTs of 500 events |
|
||||
| Provisioning (`EnsureTags`) | numeric types only; others skipped | `MERGE` upsert of every request, all types | Upsert into `{prefix}tag`, all types |
|
||||
| Connection | `Endpoint` + `ApiKey` + TLS keys (top-level `ServerHistorian:*`) | `ServerHistorian:SqlServer:*` | `ServerHistorian:QuestDb:*` |
|
||||
|
||||
@@ -1411,7 +1411,11 @@ with the first HistoryUpdate after it.
|
||||
`INSERT … SELECT … WHERE NOT EXISTS` through the `SampleRow` TVP. A null timestamp is stamped now;
|
||||
`ServerTimeUtc = SYSUTCDATETIME()`. It returns `false` (never throws) on a SQL failure so the recorder's retry/quarantine
|
||||
owns it. Semantics: [delivery semantics](#outbox-format-v2-and-rollback).
|
||||
- **Alarm events** (`IAlarmHistorianWriter`): per event, insert-if-absent keyed by a deterministic `EventId` = lowercase-hex
|
||||
- **Alarm events** (`IAlarmHistorianWriter`): a drain batch is written in **multi-row statements of 150 events** (one round
|
||||
trip per chunk — it used to be a fresh connection and a single-row statement per event, so a 10 000-event alarm storm cost
|
||||
10 000 round trips). A chunk that fails with a data error (or a duplicate-key race with the peer node) is re-sent **event by
|
||||
event**, so only the poison event dead-letters and its siblings still `Ack`; a transient failure retries the whole chunk. The
|
||||
shared orchestration is `Sql.Common.AlarmBatchWrite` (QuestDB uses it with 500-event INSERTs). Each event: insert-if-absent keyed by a deterministic `EventId` = lowercase-hex
|
||||
SHA-256 over the length-prefixed fields `EquipmentPath, AlarmId, EventKind, TimestampUtc.Ticks, User, Comment, Message`.
|
||||
Outcomes: `Ack` on insert or already-present (including a concurrent writer winning a 2627/2601 race); `RetryPlease` on
|
||||
transient SQL errors (connection, timeout, deadlock 1205 …), bootstrap failures, cancellation and any unclassified error;
|
||||
|
||||
Reference in New Issue
Block a user