fix(siteeventlog): scope the storage cap to site_events' own pages (register rows 81, 84)
Row 81 — the cap could not be enforced on a replicated node. MaxStorageMb was compared against page_count-freelist_count over the WHOLE consolidated LocalDb file, which since Phase 1/2 holds ten tables plus __localdb_oplog / __localdb_row_version / __localdb_snapshot_state. Every 1000-row trim batch freed ~390 KB of site_events pages and added ~1000 oplog rows + ~1000 tombstones to the same file, so the measured size did not fall, the "size did not decrease" guard fired, and the trim stopped 2.5x over a 64 MB cap with 226 k rows still present. A node whose peer was down could not enforce its cap at all, and the event log's budget was in practice shared with bookkeeping it does not own. The cap now measures site_events AND ITS FOUR INDEXES only, so replication bookkeeping can neither consume nor block the budget — and the cap stops being a function of the peer's health, which was the wrong direction of coupling: an inflated oplog is a replication fault, and letting it drive event deletion would destroy operational history in response to one. dbstat — the exact per-table measurement the register proposed — is NOT usable here, verified before implementing. It is a compile-time option and the bundled SQLitePCLRaw.lib.e_sqlite3 (SQLite 3.53.3) does not enable it: PRAGMA compile_options lists no ENABLE_DBSTAT_VTAB, pragma_module_list returns ten modules with neither dbstat nor sqlite_dbpage, and selecting from dbstat fails with "no such table". Switching SQLite builds was rejected — the provider is shared with ZB.MOM.WW.LocalDb and every other consumer, so it trades cap accuracy for fleet-wide version skew. The footprint is therefore modelled from sampled row width (500 rows each end, octet_length so multi-byte UTF-8 is not under-counted) through a page-layout model; whole pages rather than a flat rate, because page fill collapses once rows approach half a page and a linear model underestimates that by ~40%. Clamped by whole-file logical size, a true upper bound that can only correct an over-estimate and never binds on a replicated node. Index pages count against the cap deliberately: measured at ~34% of the table's footprint, and the 1 GB default was sized against a 393 B/row figure that already included them. Guard semantics restated — two backstops replace the single on-disk check: (1) row width unsampleable => refuse to trim (checked before the cap comparison, since a zero width makes "over cap" unanswerable rather than false; the old loop's answer to an unanswerable cap was to keep deleting); (2) estimated size not falling across a batch => stop. (2) is the old guard re-based onto the table-scoped metric, so it can no longer be tripped by replication bookkeeping and now means exactly one thing: events are arriving as fast as the trim removes them. For that to be observable the loop re-reads the row count each batch, while bytes-per-row is sampled once per tick and held fixed — re-sampling per batch would jitter the estimate as the trim eats the head of the table and could stall the loop through guard (2), the same self-defeating measurement as row 81 itself. Row 84 — LogReplicationBacklogNote moved from per-phase to once per tick, driven by the shared PurgeBudget. A tick where retention AND the cap trim both deleted used to emit two identical Information notes; the tick total is also the more useful number beside an oplog spike. Measured model error: -0.8% / +8.4% / +12.8% across representative workloads, biased high on purpose (an early cap forfeits budget; a low one fails at its only job). Independently cross-checks row 34's harness at 397.2 vs 393 B/row. Fail-before: reverting the estimator to whole-file semantics fails 5 of the new tests and reproduces the original signature exactly (remaining=2000 estimate=21725184 cap=11534336 — rows destroyed, cap still unmet); restoring per-phase notes fails the row-84 test with "collection contained 2 matching items". No config keys added or renamed, so no rig/deploy changes. Decision record: docs/plans/2026-08-17-site-event-log-cap-semantics-decision.md Tests: SiteEventLogging 104/104 (was 93), Host 520/520, solution build clean.
This commit is contained in:
@@ -4,7 +4,34 @@ public class SiteEventLogOptions
|
||||
{
|
||||
/// <summary>Number of days to retain site event log entries before purge; default 30.</summary>
|
||||
public int RetentionDays { get; set; } = 30;
|
||||
/// <summary>Maximum SQLite database size in megabytes before old entries are purged; default 1024 MB.</summary>
|
||||
/// <summary>
|
||||
/// Maximum size in megabytes of <c>site_events</c> AND ITS FOUR INDEXES before the oldest
|
||||
/// entries are trimmed; default 1024 MB.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>This is not a cap on the database file</b> (register row 81, decision record
|
||||
/// <c>docs/plans/2026-08-17-site-event-log-cap-semantics-decision.md</c>). <c>site_events</c>
|
||||
/// shares one SQLite file at <c>LocalDb:Path</c> with nine other tables plus LocalDb's
|
||||
/// <c>__localdb_oplog</c> / <c>__localdb_row_version</c> bookkeeping. Measuring the whole file
|
||||
/// made the trim unwinnable on a replication-enabled node — each batch of deletes generated
|
||||
/// an equal number of oplog rows and tombstones in the same file, so the measured size never
|
||||
/// fell and the trim stopped 2.5x over cap. Worse, it made the event log's budget a function
|
||||
/// of the replication peer's health, in the direction where a replication fault destroys
|
||||
/// operational history.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Consequence for sizing a site's data volume: this value alone does not bound it. LocalDb
|
||||
/// replication bookkeeping is separate overhead on the same disk, bounded by
|
||||
/// <c>LocalDb:Replication:MaxOplogRows</c> and <c>TombstoneRetention</c>. Budget the sum.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The footprint is MODELLED from sampled row width rather than measured exactly: SQLite's
|
||||
/// <c>dbstat</c> virtual table would give an exact per-table page count but is not compiled
|
||||
/// into the bundled native library. Measured accuracy is −0.8 % to +12.8 %, deliberately
|
||||
/// biased high. See <c>EventLogPurgeService.EstimateSiteEventsSize</c>.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public int MaxStorageMb { get; set; } = 1024;
|
||||
/// <summary>
|
||||
/// MIGRATION-ONLY as of LocalDb Phase 1. <see cref="SiteEventLogger"/> no longer reads
|
||||
|
||||
Reference in New Issue
Block a user