perf(historian): recorder appends off its mailbox; Periodic commit default (lmxopcua#746)
The continuous-historization recorder awaited every outbox append (an fsync under PerEntry) on its mailbox, so ingestion was capped at 1/fsync_latency bursts per second while the mux kept telling into an unbounded, unmetered mailbox. GetStatus also suspended the mailbox on an unbounded outbox count. - Gated ingest bursts go to a bounded channel (64 bursts) drained by one ordered append loop, which commits every burst waiting behind a slow append in a single AppendBatchAsync (FIFO kept). Only a full queue holds the mailbox (backpressure, bounded memory). Counters are Interlocked; the loop nudges the drain. PostStop flushes queued bursts within the shared 5 s stop budget. - GetStatus reads counters on the actor thread and pipes a count bounded by a 10 s timeout (-1 when it does not answer); it no longer suspends the mailbox. RecorderStatus gains PendingAppends, MailboxDepth and AppendBackpressureWaits; two gauges publish pending appends and mailbox depth. - ContinuousHistorization:CommitMode defaults to Periodic (100 ms). The Host's appsettings.json, its unparseable-value fallback and docker-dev still say PerEntry: filed as a Host follow-up (outside this lane). Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22 in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -38,22 +38,20 @@ public sealed class ContinuousHistorizationOptions
|
||||
public string OutboxPath { get; init; } = "";
|
||||
|
||||
/// <summary>
|
||||
/// Outbox commit cadence: <c>PerEntry</c> (the default) fsyncs the log before each append
|
||||
/// returns (safest, no loss window); <c>Periodic</c> batches commits onto a background timer
|
||||
/// every <see cref="CommitIntervalMs"/> ms (higher throughput, a bounded worst-case loss window).
|
||||
/// Parsed case-insensitively against <c>HistorizationCommitMode</c>; an unrecognized value falls
|
||||
/// back to <c>PerEntry</c>.
|
||||
/// Outbox commit cadence: <c>Periodic</c> (the default) commits on a background timer every
|
||||
/// <see cref="CommitIntervalMs"/> ms (100 ms by default — a crash can lose at most the last interval of captured
|
||||
/// values); <c>PerEntry</c> fsyncs before each append returns (no loss window, one fsync per append). Parsed
|
||||
/// case-insensitively against <c>HistorizationCommitMode</c>.
|
||||
/// <para>
|
||||
/// <b><c>PerEntry</c> now means one fsync per ingest <em>burst</em>, not per value.</b> The
|
||||
/// recorder appends a whole value burst through
|
||||
/// <c>IHistorizationOutbox.AppendBatchAsync</c>, which enqueues every entry and then issues a
|
||||
/// <em>single</em> group commit — so a burst of N values costs one fsync instead of N. The
|
||||
/// durability contract is unchanged: <c>AppendBatchAsync</c> does not return until that
|
||||
/// commit completes, so nothing the caller has been told is buffered can be lost to an
|
||||
/// unclean restart.
|
||||
/// <b>Why Periodic.</b> Under <c>PerEntry</c> every ingest append waits for an fsync, so ingestion is capped at
|
||||
/// <c>1 / fsync_latency</c> appends per second (50 drivers × 1 Hz on a 10 ms-fsync disk kept the recorder half
|
||||
/// busy; at 100+ bursts/s it fell behind for good). The recorder now appends off its mailbox and coalesces
|
||||
/// every burst waiting behind a slow append into one group commit, which helps <c>PerEntry</c> too — but it is
|
||||
/// still one fsync per group on the critical path. <c>PerEntry</c> remains available where no loss window is
|
||||
/// acceptable.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public string CommitMode { get; init; } = "PerEntry";
|
||||
public string CommitMode { get; init; } = "Periodic";
|
||||
|
||||
/// <summary>Periodic-mode commit cadence in milliseconds; must be positive when
|
||||
/// <see cref="CommitMode"/> is <c>Periodic</c>. Ignored under <c>PerEntry</c>.</summary>
|
||||
|
||||
Reference in New Issue
Block a user