feat(historian): SQL Server retention purge (RetentionDays, Primary-gated, applock) (#667)

docs/Configuration.md

@@ -411,6 +411,9 @@ Validated at host start (driver-role nodes) when `ServerHistorian:Enabled` **or*
 | `ServerHistorian:SqlServer:Schema` | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (spliced into DDL/DML). |
 | `ServerHistorian:SqlServer:CommandTimeout` | `00:00:30` | Must be `> 0`; rounded up to whole seconds. |
 | `ServerHistorian:SqlServer:AutoCreateSchema` | `true` | `true` applies the embedded schema script under an app-lock on first use; `false` only verifies `SchemaVersion >= 1` and fails every historian call (health `LastError`, no crash) if behind. |
+| `ServerHistorian:SqlServer:RetentionDays` | unset | Days of history kept (source time), `>= 1` when set; unset keeps everything. Enables the Primary-gated purge of `hist.Sample`/`hist.Event` (see [Historian.md](Historian.md#retention-retentiondays)). |
+| `ServerHistorian:SqlServer:PurgeInterval` | `01:00:00` | Purge cadence; must be `> 0`. |
+| `ServerHistorian:SqlServer:PurgeBatchSize` | `4000` | Rows per `DELETE TOP (n)` batch; `>= 1`. Keep under ~5000 to avoid lock escalation. |
 
 #### `ServerHistorian:QuestDb` (`Backend=QuestDb`)
 

docs/Historian.md

@@ -151,6 +151,9 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
 | `SqlServer:Schema` | string | `hist` | Must match `^[A-Za-z_][A-Za-z0-9_]{0,127}$` (it is spliced into DDL/DML) | SQL schema that holds `SchemaVersion`, `Tag`, `Sample`, `Event` and the `SampleRow`/`TimeRow` table types. |
 | `SqlServer:CommandTimeout` | TimeSpan | `00:00:30` | Must be `> 0` | Per-command timeout, rounded up to whole seconds (minimum 1 s). The SQL backend does **not** use `CallTimeout`. |
 | `SqlServer:AutoCreateSchema` | bool | `true` | — | `true`: on first use apply any missing embedded script under a `sp_getapplock` (`OtOpcUaHistorianSchema`), so concurrent nodes serialise. `false`: only verify `SchemaVersion >= 1`; if behind, **every** historian call fails with a clear error naming the missing version (surfaced in the read client's health `LastError`, never a host crash). A failed bootstrap is retried on the next call, not latched. |
+| `SqlServer:RetentionDays` | int? | unset | `>= 1` when set | Days of history kept, on **source time**. Unset keeps everything. When set, the Primary driver node purges older `hist.Sample` / `hist.Event` rows — see [Retention](#retention-retentiondays). |
+| `SqlServer:PurgeInterval` | TimeSpan | `01:00:00` | `> 0` | How often the purge runs (first pass at start-up). |
+| `SqlServer:PurgeBatchSize` | int | `4000` | `>= 1` | Rows per `DELETE TOP (n)`; the default stays under SQL Server's ~5000-lock escalation threshold so a purge never takes a table lock. |
 
 **`ServerHistorian:QuestDb`** (read only when `Backend=QuestDb`; bound by `HistorianBackendRegistry.BindQuestDb`): six keys — `ConnectionString`
 (required), `TablePrefix`, `RetentionDays`, `CommandTimeout`, `AutoCreateSchema`, `VisibilityTimeout`. The full table, defaults and validation
@@ -772,6 +775,27 @@ The read client reports `GetHealthSnapshot()` (query/failure counters, consecuti
 `AutoCreateSchema=false`). A SQL error on a read surfaces as `BadHistoryOperationUnsupported` for **that node only**; a cancelled read
 (deadline, client abort) is not counted as a backend fault.
 
+### Retention (`RetentionDays`)
+
+`hist.Sample` and `hist.Event` grow without bound unless `ServerHistorian:SqlServer:RetentionDays` is set. QuestDB already has an
+equivalent TTL (`QuestDb:RetentionDays`); the Gateway backend's retention is the AVEVA Historian's own.
+
+- **What runs.** `HistorianRetentionService` (a hosted service, driver-role nodes, Backend=`SqlServer` with `RetentionDays` set; inert
+  otherwise) ticks every `PurgeInterval` (first pass at start-up) and calls the backend's `IHistorianRetention.PurgeAsync`.
+- **Primary-gated.** Each tick consults `IHistorizationRoleView.ShouldRecordHistorizedValues` (the same verdict that gates capture); a
+  standby skips, an unknown role runs.
+- **How it deletes.** Cutoff = `UtcNow - RetentionDays` on source time. Samples are purged **per tag** with
+  `DELETE TOP (@n) ... WHERE TagId = @id AND SourceTimeUtc < @cutoff` (a seek on the clustered `(TagId, SourceTimeUtc)` key; a global
+  time predicate would scan), then events with `DELETE TOP (@n) ... WHERE EventTimeUtc < @cutoff` (seek on `CX_Event_Time`), each looped
+  until a batch comes back short. Cancellation is honoured between batches. The `hist.Tag` catalogue is never purged.
+- **Shared databases.** Several pairs may point at one database; a session `sp_getapplock` (`OtOpcUa.Historian.Retention.<schema>`,
+  timeout 0) lets one purger run and the others skip that tick (Debug log).
+- **Observability.** Counter `otopcua.historian.retention.purged{table=Sample|Event}`; a failed pass logs a Warning and retries next
+  interval — it never affects capture or reads.
+- **Not done: partition switching.** For very large installs (billions of rows) a partitioned `hist.Sample` with sliding-window
+  `SWITCH`/drop is cheaper than row deletes; the purger is deliberately simple batched deletes. Another table (e.g. a future
+  `hist.SampleModified`, #668) is added by one more `DeleteLoopAsync` call in `SqlServerHistorianRetention.PurgeAsync`.
+
 ### Docker-dev recipe
 
 The docker-dev `sql` container (`otopcua-dev-sql-1`, host port `14330`, `sa` with the committed dev password from
@@ -795,7 +819,9 @@ sqlcmd -S localhost,14330 -U sa -P 'OtOpcUa!Dev123' -C -I -d OtOpcUaHistorian \
   -Q "SELECT t.Name, COUNT(*) n, MAX(s.SourceTimeUtc) last FROM hist.Sample s JOIN hist.Tag t ON t.TagId=s.TagId GROUP BY t.Name"
 ```
 
-Only the Primary of a pair records, so the row count advances once per published value, not twice. Read it back with
+Only the Primary of a pair records, so the row count advances once per published value, not twice. To watch retention, add
+`ServerHistorian__SqlServer__RetentionDays: "1"` and `ServerHistorian__SqlServer__PurgeInterval: "00:00:30"`, insert a `hist.Sample` row
+with an old `SourceTimeUtc` for an existing tag, and see it vanish within a tick while recent rows stay. Read it back with
 `Client.CLI historyread` and rewrite it with `Client.CLI historyupdate` ([Client.CLI.md](Client.CLI.md)). The password above is the
 committed docker-dev credential, not a secret; use `${secret:…}` / environment for anything real.