docs(kpihistory): document hourly rollups + resolve deferred #22 (plan #22 T8)

Update Component-KpiHistory.md with the KpiRollupHourly schema, the third
recorder rollup tick + one-shot backfill, dual raw/rollup purge retention,
raw-vs-rollup query routing by RollupThresholdHours, per-metric gauge-vs-rate
aggregation (KpiMetricAggregationCatalog), the four new options + bounds, and
30d/90d trend windows. Append a Delivered-2026-07-10 note to the m6 design
plan, move register row #22 from Deferred to Resolved, and note rollups in the
CLAUDE.md KPI History bullet. Docs-only; no code changed.

Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
This commit is contained in:
Joseph Doherty
2026-07-10 12:18:46 -04:00
parent bbfc1b3278
commit c9fa3b07f4
4 changed files with 72 additions and 16 deletions
+1 -1
View File
@@ -212,7 +212,7 @@ Other peers in the `scadaproj` family (see `scadaproj/CLAUDE.md` for details): `
- Stuck = `Pending`/`Retrying` older than a configurable age threshold (default 10 min) — display-only (KPI count + row badge), no escalation/alerting.
- Headline KPI tiles surface on the Health dashboard; a new Central UI Notification Outbox page offers a queryable list with Retry/Discard actions on parked notifications.
- Site Call Audit KPIs are central-computed point-in-time from the `SiteCalls` table (global + per-site), mirroring the Notification Outbox KPI shape; tiles surface on the Health dashboard alongside a queryable Central UI Site Calls page with Retry/Discard on parked rows.
- KPI History & Trends (#26, M6): a reusable central KPI-history backbone — supersedes the prior "point-in-time only, no time-series store" stance — backed by a tall/EAV `KpiSample` table in central MS SQL (no new infra). A `KpiHistoryRecorderActor` cluster singleton (`kpi-history-recorder`, **not readiness-gated**, best-effort with per-source isolation) samples every minute by enumerating DI-registered `IKpiSampleSource`s (each lives with its owner, registered via `TryAddEnumerable`, reusing existing KPI/aggregator reads); daily purge after `RetentionDays` (default 90). Querying is `IKpiHistoryRepository.GetRawSeriesAsync``KpiSeriesBucketer` (last-value-per-bucket) → scoped dual-ctor `KpiHistoryQueryService` → a reusable **custom-SVG** `KpiTrendChart` (no third-party charting lib). Trends ship on four surfaces: Notification Outbox, Site Calls, Audit Log pages + a per-site Health-dashboard panel. `KpiHistoryOptions` (`ScadaBridge:KpiHistory`): SampleInterval 60s, RetentionDays 90, PurgeInterval 1d, DefaultMaxSeriesPoints 200; validated. M6's T9 and T10 were originally deferred. **T9/T10 delivered 2026-06-19** as an SMS (Twilio) adapter — Teams was evaluated and dropped (outbound-only backends cannot send 1:1 chat bodies via Graph without a Bot Framework inbound endpoint; SMS is inherently per-person and outbound-only). `NotificationType.Sms` added; `SmsNotificationDeliveryAdapter` (Twilio REST, no SDK) registered alongside Email; Central UI Type selector (adapter-gated) + SMS recipient input; CLI `--type sms --phones` on list create/update; `notification sms list|update` commands.
- KPI History & Trends (#26, M6): a reusable central KPI-history backbone — supersedes the prior "point-in-time only, no time-series store" stance — backed by a tall/EAV `KpiSample` table in central MS SQL (no new infra). A `KpiHistoryRecorderActor` cluster singleton (`kpi-history-recorder`, **not readiness-gated**, best-effort with per-source isolation) samples every minute by enumerating DI-registered `IKpiSampleSource`s (each lives with its owner, registered via `TryAddEnumerable`, reusing existing KPI/aggregator reads); daily purge after `RetentionDays` (default 90). Querying is `IKpiHistoryRepository.GetRawSeriesAsync``KpiSeriesBucketer` (last-value-per-bucket) → scoped dual-ctor `KpiHistoryQueryService` → a reusable **custom-SVG** `KpiTrendChart` (no third-party charting lib). Trends ship on four surfaces: Notification Outbox, Site Calls, Audit Log pages + a per-site Health-dashboard panel. `KpiHistoryOptions` (`ScadaBridge:KpiHistory`): SampleInterval 60s, RetentionDays 90, PurgeInterval 1d, DefaultMaxSeriesPoints 200; validated. **Hourly rollups shipped 2026-07-10 (deferred #22):** a second `KpiRollupHourly` table folded by a third recorder tick (`kpi-rollup`, `RollupInterval` 1h, re-folds `RollupLookbackHours` window via idempotent failover-self-healing upsert + one-shot backfill on start), per-metric gauge-vs-rate fold (`KpiMetricAggregationCatalog`), and raw-vs-rollup query routing by `RollupThresholdHours` (168h) unblock long-range **30d/90d** trend windows (rollups retained `RollupRetentionDays` 365 ≥ raw 90, dual daily purge). M6's T9 and T10 were originally deferred. **T9/T10 delivered 2026-06-19** as an SMS (Twilio) adapter — Teams was evaluated and dropped (outbound-only backends cannot send 1:1 chat bodies via Graph without a Bot Framework inbound endpoint; SMS is inherently per-person and outbound-only). `NotificationType.Sms` added; `SmsNotificationDeliveryAdapter` (Twilio REST, no SDK) registered alongside Email; Central UI Type selector (adapter-gated) + SMS recipient input; CLI `--type sms --phones` on list create/update; `notification sms list|update` commands.
### Code Organization
- Entity classes are persistence-ignorant POCOs in Commons; EF mappings in Configuration Database.
@@ -95,6 +95,13 @@ skips that source's samples for the tick — it never throws into or disrupts th
`RetentionDays` (default 90), reusing the existing purge-scheduler shape. Hourly downsampling
beyond N days is deferred (YAGNI).
> **Delivered 2026-07-10** (deferred-work register #22): the hourly-downsampling deferral is
> closed. A `KpiRollupHourly` table folded by a third recorder tick, per-metric gauge-vs-rate
> aggregation, raw-vs-rollup query routing by `RollupThresholdHours`, longer rollup retention,
> a one-shot backfill, and 30 d / 90 d trend windows shipped per
> `docs/plans/2026-07-10-kpi-history-hourly-rollups-plan.md`. See `Component-KpiHistory.md` for
> the shipped design.
### Sample sources
- **`NotificationOutboxKpiSampleSource`** (in NotificationOutbox) →
@@ -26,7 +26,6 @@ them and are removed from this table when that plan's task lands.
| 17 | Unified notifications+site-calls outbox page | stillpending :118 | Explicit M9 decision to keep two pages | Operator confusion reports |
| 18 | Folder drag-drop | same, [PERM] | Permanently closed; menu reorder shipped | — (closed) |
| 19 | Bundle signing / cluster-to-cluster pull / differential bundles | transport-design :402 | v1 manifest hash + AES-GCM held sufficient | Non-repudiation requirement across orgs |
| 22 | KPI history hourly rollups | Component-KpiHistory.md | YAGNI; 90-day retention bounds table | KpiSample query latency on dashboards |
## Resolved (verified against the code 2026-07-10)
Rows removed from the Deferred table above once confirmed shipped. Kept here for traceability.
@@ -41,6 +40,7 @@ Rows removed from the Deferred table above once confirmed shipped. Kept here for
| 16 | StubOpcUaClient throws on browse | Already resolved: `StubOpcUaClient` supports browse + address-space search, covered by `StubOpcUaClientBrowseTests`/`StubOpcUaClientSearchTests`. |
| 20 | Deployment EXPIRED-row purge | Already resolved (PLAN-04): `PendingDeploymentPurgeActor` central singleton (spawned in `AkkaHostedService`) ticks `IDeploymentManagerRepository.PurgeExpiredPendingDeploymentsAsync` every `CommunicationOptions.PendingDeploymentPurgeInterval` (default 1h), options-validated, tested. |
| 21 | SiteAuditBacklogReporter threshold consolidation | Shipped 2026-07-10: `SqliteAuditWriterOptions.BacklogPollIntervalSeconds` (default 30) now drives the reporter's poll cadence; explicit ctor override still wins (tests), non-positive falls back to the 30 s default. Cadence tests added; stale "hard-code / follow-up" class-doc corrected. |
| 22 | KPI history hourly rollups | Shipped 2026-07-10 (`docs/plans/2026-07-10-kpi-history-hourly-rollups-plan.md`, T1T8): new `KpiRollupHourly` table (migration `20260710153953`) folded by a third recorder tick (`kpi-rollup`, `RollupInterval` default 1h) over a re-folded `RollupLookbackHours` window via an idempotent, failover-self-healing upsert; per-metric gauge-vs-rate aggregation (`KpiMetricAggregationCatalog`); a one-shot backfill of the retention window on start; raw-vs-rollup query routing by `RollupThresholdHours` (default 168h); longer rollup retention (`RollupRetentionDays` default 365 ≥ `RetentionDays`, dual daily purge); and 30 d / 90 d trend windows added to the four surfaces. Options + validator, docs (`Component-KpiHistory.md`), and tests shipped. |
## New deferrals from review 08 (this plan)
| Item | Rationale | Revisit trigger |
+63 -14
View File
@@ -4,7 +4,7 @@
The KPI History component is the central, reusable **KPI-history backbone** — a tall / EAV time-series store, a periodic recorder singleton, a bucketed query API, and a reusable custom SVG trend-chart component. It turns the system's existing point-in-time KPIs into trends, and ships those trends for the **Notification Outbox (#21)**, **Site Call Audit (#22)**, **Audit Log (#23)**, and **Site Health (#11)** sources.
This supersedes the earlier "KPI history — point-in-time only, no separate time-series store is added" stance carried by the Notification Outbox and Site Call Audit KPI sections. M6 explicitly introduces a store. It lives in **central MS SQL** — the existing HA store — so it adds **no new infrastructure dependency**: a single `KpiSample` table, an EF mapping + migration, and a central cluster singleton that samples every minute.
This supersedes the earlier "KPI history — point-in-time only, no separate time-series store is added" stance carried by the Notification Outbox and Site Call Audit KPI sections. M6 explicitly introduces a store. It lives in **central MS SQL** — the existing HA store — so it adds **no new infrastructure dependency**: a raw `KpiSample` table plus an hourly `KpiRollupHourly` rollup table (the latter added for the deferred #22 long-range-query slice), EF mappings + migrations, and a central cluster singleton that samples every minute and folds hourly rollups.
The backbone is deliberately source-agnostic. Each owning component contributes an `IKpiSampleSource` registered into DI; the recorder enumerates them. KPI History therefore does **not** reference every component, and every source reuses the KPI reads its owner already computes — no per-source schema or storage work.
@@ -12,19 +12,20 @@ The backbone is deliberately source-agnostic. Each owning component contributes
- `src/ZB.MOM.WW.ScadaBridge.KpiHistory` — the component project: the `KpiHistoryRecorderActor`, `KpiHistoryOptions` + validator, and the DI/options wiring (`ServiceCollectionExtensions`). It owns the recorder, the options, and consumes the `IKpiSampleSource` abstraction (defined in Commons).
- **`IKpiSampleSource` implementations live with their owners**, not here — `NotificationOutboxKpiSampleSource` (in NotificationOutbox), `SiteCallAuditKpiSampleSource` (in SiteCallAudit), `AuditLogKpiSampleSource` (in AuditLog), `SiteHealthKpiSampleSource` (in HealthMonitoring). Each registers itself via `TryAddEnumerable`.
- **Commons** — the `KpiSample` POCO entity (`Entities/Kpi`), the `IKpiSampleSource` and `IKpiHistoryRepository` interfaces (`Interfaces/Kpi`), and the `KpiSources` / `KpiScopes` constant catalogs + `KpiSeriesPoint` / `KpiSeriesBucketer` types (`Types/Kpi`).
- **Configuration Database** — the EF mapping (`KpiSampleEntityTypeConfiguration`), the migration that creates the `KpiSample` table + indexes, and the `KpiHistoryRepository` implementation.
- **Commons** — the `KpiSample` and `KpiRollupHourly` POCO entities (`Entities/Kpi`), the `IKpiSampleSource` and `IKpiHistoryRepository` interfaces (`Interfaces/Kpi`), and the `KpiSources` / `KpiScopes` constant catalogs + `KpiSeriesPoint` / `KpiSeriesBucketer` types and the `KpiRollupAggregation` enum + `KpiMetricAggregationCatalog` (`Types/Kpi`).
- **Configuration Database** — the EF mappings (`KpiSampleEntityTypeConfiguration`, `KpiRollupHourlyEntityTypeConfiguration`), the migrations that create the `KpiSample` and `KpiRollupHourly` tables + indexes, and the `KpiHistoryRepository` implementation (raw record/query/purge plus the hourly fold / rollup query / rollup purge).
- **Central UI** — the `KpiHistoryQueryService` query service and the reusable `KpiTrendChart.razor` component, plus the trend sections embedded on four surfaces.
The recorder is a **singleton on the active central node**, consistent with the other central singletons (Notification Outbox, Site Call Audit, purge actors).
## Responsibilities
- Own the `KpiSample` table — the central tall / EAV KPI-history store in MS SQL.
- Own the `KpiSample` and `KpiRollupHourly` tables — the central tall / EAV KPI-history store (raw + hourly rollups) in MS SQL.
- Run the recorder loop: every `SampleInterval`, enumerate all registered `IKpiSampleSource`s and persist their samples stamped with one shared tick timestamp.
- Isolate sources from one another and from the store: a failure in any one source (or in the write) is logged and skipped for that tick and never disrupts the source component or the rest of the tick (best-effort observability).
- Purge aged rows on a daily cadence (`PurgeInterval`) older than `RetentionDays`.
- Provide a bucketed series-query API (`IKpiHistoryRepository.GetRawSeriesAsync` + `KpiSeriesBucketer`) and the Central UI query service + reusable trend chart that consume it.
- Fold raw samples into hourly rollups on a `RollupInterval` cadence (with a one-shot backfill on start), applying per-metric gauge-vs-rate aggregation via an idempotent, failover-self-healing upsert.
- Purge aged rows on a daily cadence (`PurgeInterval`): raw rows older than `RetentionDays` and rollup rows older than the longer `RollupRetentionDays`.
- Provide a bucketed series-query API (`IKpiHistoryRepository.GetRawSeriesAsync` / `GetHourlySeriesAsync` + `KpiSeriesBucketer`, routed raw-vs-rollup by window width) and the Central UI query service + reusable trend chart that consume it.
KPI History is **observability, never a user-facing critical path** — neither recording nor querying may ever break a hosting page or disrupt a source component.
@@ -49,6 +50,32 @@ Two named indexes back the access paths:
- **`IX_KpiSample_Series` (`Source`, `Metric`, `Scope`, `ScopeKey`, `CapturedAtUtc`)** — the per-series range query (one series scanned in time order).
- **`IX_KpiSample_Captured` (`CapturedAtUtc`)** — the retention purge.
## Schema — `KpiRollupHourly` (hourly rollups)
Long-range trend queries (30 d / 90 d) would fetch and sort tens of thousands of raw `KpiSample` rows per series (~43k at 30 d, ~130k at 90 d @ 1/min) before bucketing. The recorder therefore pre-aggregates raw samples into an **hourly rollup** table, `KpiRollupHourly` — a second tall / EAV table in the same central MS SQL database (no new infrastructure), keyed by the **identical four-tuple series key** plus a truncated `HourStartUtc`. A 90 d read becomes a small indexed scan (~2,160 rows/series vs ~130k). One row per `(Source, Metric, Scope, ScopeKey)` per UTC hour:
| Column | Type | Notes |
|---|---|---|
| `Id` | `bigint` PK identity | Surrogate key assigned by the store. |
| `Source` | `varchar(64)` | Same `KpiSources` constant as `KpiSample`. |
| `Metric` | `varchar(64)` | Same per-source metric name as `KpiSample`. |
| `Scope` | `varchar(16)` | A `KpiScopes` constant: `Global` / `Site` / `Node`. |
| `ScopeKey` | `varchar(64)` NULL | Site id / node name; `NULL` for `Global`. |
| `HourStartUtc` | `datetime2` | Hour-truncated UTC bucket start (minutes/seconds/ticks zeroed). |
| `Value` | `float` (`double`) | The folded aggregate — a **sum-per-hour** for rate metrics or a **last-value-per-hour** for gauge metrics (see Query — aggregation intent). |
| `MinValue` | `float` (`double`) | Minimum raw sample value observed within the hour. |
| `MaxValue` | `float` (`double`) | Maximum raw sample value observed within the hour. |
| `SampleCount` | `int` | Number of raw `KpiSample` rows folded into the hour. |
`MinValue` / `MaxValue` / `SampleCount` preserve the fold's fidelity and unblock future min/max/avg charting; the current `KpiTrendChart` still plots the single `Value` series (see Query).
Two named indexes back the access paths, mirroring `KpiSample`:
- **`IX_KpiRollupHourly_Series` (`Source`, `Metric`, `Scope`, `ScopeKey`, `HourStartUtc`)** — the UNIQUE index that is **both** the idempotent upsert key and the covering index for the bucketed range read. It is declared with `HasFilter(null)` to **suppress EF's default `[ScopeKey] IS NOT NULL` filtered index**, so `Global`-scope rows (null `ScopeKey`) participate in the uniqueness constraint and the per-`(series, hour)` upsert holds for global rollups too.
- **`IX_KpiRollupHourly_Hour` (`HourStartUtc`)** — the retention purge.
The table is non-partitioned, on the standard `[PRIMARY]` filegroup, with no DB-role restriction — operational history, not audit (mirrors `KpiSample`). Entity + EF mapping in Commons / Configuration Database; created by migration `20260710153953_AddKpiRollupHourlyTable`.
## Recorder — `KpiHistoryRecorderActor`
The recorder is the Akka.NET cluster singleton **`kpi-history-recorder`** (singleton-manager actor `kpi-history-recorder-singleton`), running on the active central node. It is **not readiness-gated** — the recorder is pure observability and must never gate `/health/ready`, so it is started outside the readiness barrier (unlike the operational singletons). On graceful shutdown it drains via a `CoordinatedShutdown` task for clean singleton handover.
@@ -61,7 +88,11 @@ A timer fires every `SampleInterval` (default 60s; an immediate first tick prime
**Best-effort, per-source isolation.** Each source call and the write are individually guarded. A throwing source is logged and its samples skipped for that tick; it never aborts the tick, the other sources, or the source component itself. This is the same `IEnumerable<>`-of-adapters decoupling pattern used by `INotificationDeliveryAdapter`.
**Retention.** A daily purge timer (`PurgeInterval`, default 24h) deletes rows older than `RetentionDays` (default 90) via `IKpiHistoryRepository.PurgeOlderThanAsync`, reusing the existing purge-scheduler shape. Hourly/longer-range downsampling is deferred (YAGNI).
**Hourly rollup tick.** A third timer (`kpi-rollup`) fires every `RollupInterval` (default 1h). On each tick the recorder opens a per-tick DI scope and calls `IKpiHistoryRepository.FoldHourlyRollupsAsync` over the window `[TruncateToHour(now) RollupLookbackHours, TruncateToHour(now))` — an **exclusive** upper bound at the current hour start, so the in-progress hour is never folded. The fold groups raw `KpiSample` rows by `(series, hour)`, applies the per-metric aggregation intent (sum for rate metrics, last-value for gauges — see Query), and **idempotently upserts** each `(Source, Metric, Scope, ScopeKey, HourStartUtc)` row. Re-folding the trailing lookback window (default 3h) rather than only the last hour is the self-heal for a **singleton-failover-missed tick**: because the upsert overwrites the aggregate in place, re-running the same window produces identical values (no double-count), so any complete hour a missed tick skipped is refilled on the next fold. A `_rollupInFlight` guard (mirroring `_sampleInFlight`) skips a tick while a prior fold is still running so two idempotent upserts never race on overlapping rows.
**One-shot backfill.** So the 30 d / 90 d charts are not blank until enough wall-clock passes, the recorder runs a **single** backfill on start (`kpi-rollup-backfill` timer) that folds the full retention window once, `[TruncateToHour(now) RetentionDays, TruncateToHour(now))`, reusing the same `FoldHourlyRollupsAsync` path. A `_backfillDone` latch makes it at-most-once per actor lifetime, and it defers (re-arms) while a periodic fold is in flight; being an idempotent upsert, it is cheap and safe to re-run on a later failover restart.
**Retention.** A daily purge timer (`PurgeInterval`, default 24h) now runs **both** purges: raw `KpiSample` rows older than `RetentionDays` (default 90) via `IKpiHistoryRepository.PurgeOlderThanAsync`, **and** hourly rollup rows older than `RollupRetentionDays` (default 365) via `IKpiHistoryRepository.PurgeRollupsOlderThanAsync` (a one-hour-sliced batched DELETE mirroring the raw purge). Rollups deliberately **outlive** raw samples so long-range trend charts have data after the raw rows are purged; the options validator enforces `RollupRetentionDays >= RetentionDays` so a window never falls into a hole where raw is purged but no rollup exists.
## Sample Sources
@@ -117,15 +148,29 @@ Reads `ICentralHealthAggregator.GetAllSiteStates()` (in-memory, no DB). Scope: *
### Bucketed query
`IKpiHistoryRepository.GetRawSeriesAsync(source, metric, scope, scopeKey, fromUtc, toUtc, …)` returns the raw points for one series over `[fromUtc, toUtc]`. `KpiSeriesBucketer.Bucket(raw, fromUtc, toUtc, maxPoints)` then partitions the window into ≤ `maxPoints` time buckets and returns the **last value per bucket** as `KpiSeriesPoint(BucketStartUtc, Value)`. Last-value is correct for gauge metrics; v1 ships exactly one aggregation — avg / min / max are deferred.
`IKpiHistoryRepository.GetRawSeriesAsync(source, metric, scope, scopeKey, fromUtc, toUtc, …)` returns the raw points for one series over `[fromUtc, toUtc]`. `GetHourlySeriesAsync(…)` returns the same ascending `KpiSeriesPoint` shape from `KpiRollupHourly` (projecting `HourStartUtc` → the point timestamp, `Value`), so the bucketer is agnostic to the source. `KpiSeriesBucketer.Bucket(series, fromUtc, toUtc, maxPoints)` then partitions the window into ≤ `maxPoints` time buckets and returns the **last value per bucket** as `KpiSeriesPoint(BucketStartUtc, Value)` — unchanged, and still applied on top of a rollup series (a 90 d rollup is ~2,160 points > the 200-point cap, so bucketing still thins it). The chart plots the single `Value` series; avg / min / max charting is still deferred, though the rollup's `MinValue` / `MaxValue` / `SampleCount` columns are now stored to unblock it.
### Raw-vs-rollup range routing
`GetRawSeriesAsync` and `GetHourlySeriesAsync` are selected by window width in the query service (below), governed by `RollupThresholdHours` (default 168 = 7 d). Windows **at or below** the threshold read raw samples (preserving intra-minute detail on the 24 h / 7 d charts); windows **strictly wider** than the threshold read the pre-aggregated hourly rollups (≈60× fewer rows scanned on the 30 d / 90 d charts). The boundary is strict — a window of exactly `RollupThresholdHours` stays on the raw path.
### Aggregation intent — gauge vs. rate
Not every metric folds the same way, and choosing wrong silently corrupts long-range totals. `KpiMetricAggregationCatalog.Resolve(source, metric)` classifies each `(Source, Metric)` pair as `KpiRollupAggregation.Gauge` (fold = **last-value-per-hour**) or `KpiRollupAggregation.Rate` (fold = **sum-per-hour**):
- **Gauge** (last-value) — instantaneous readings where a per-hour sum is meaningless: `queueDepth`, `buffered`, `stuck`/`stuckCount`, `parked*`, `oldestPendingAgeSeconds`, `connectionsUp`/`connectionsDown`, `deployedInstances`, and every other metric not explicitly listed as a rate. This is the **safe default**: an unknown / newly-added metric resolves to Gauge and never crashes the fold or invents a total.
- **Rate** (sum-per-hour) — already-windowed per-interval deltas whose hour total is the sum of its minute samples: `NotificationOutbox/deliveredLastInterval`, `SiteCallAudit/failedLastInterval`, `SiteCallAudit/deliveredLastInterval`, and the per-report `SiteHealth` error counts `scriptErrors` / `alarmEvalErrors` / `deadLetters` / `eventLogWriteFailures`. Folding these as last-value would keep one minute's delta and discard the other 59, under-counting every long-range total.
- **Deliberately Gauge, not Rate** — `AuditLog/totalEventsLastHour` and `errorEventsLastHour` read as rate-like but are computed as a **rolling trailing-one-hour** count re-evaluated every capture; consecutive minutes overlap by 59 minutes, so summing would over-count ~60×. Last-value is correct.
The catalog is keyed by `(source, metric)` — not metric alone — because `deliveredLastInterval` is emitted by both Notification Outbox and Site Call Audit, so a metric name is not globally unique.
### `KpiHistoryQueryService` (Central UI)
A scoped-repository direct read with a **dual-constructor test seam** (one ctor resolves a scoped `IKpiHistoryRepository` per call; the other accepts an injected repository for tests) — the same shape as `AuditLogQueryService`. `GetSeriesAsync` resolves the effective point cap (caller override or `KpiHistoryOptions.DefaultMaxSeriesPoints`), fetches the raw series, and reduces it via `KpiSeriesBucketer`. A query failure surfaces as an unavailable chart (em-dash / message), mirroring how the existing KPI tiles surface transient failures — it never breaks the hosting page.
A scoped-repository direct read with a **dual-constructor test seam** (one ctor resolves a scoped `IKpiHistoryRepository` per call; the other accepts an injected repository for tests) — the same shape as `AuditLogQueryService`. `GetSeriesAsync` resolves the effective point cap (caller override or `KpiHistoryOptions.DefaultMaxSeriesPoints`), fetches the series — **routing raw vs. hourly-rollup by window width** against `RollupThresholdHours` (see Raw-vs-rollup range routing, above) — and reduces it via `KpiSeriesBucketer`. The callers pass `fromUtc` / `toUtc` unchanged; the long-window buttons transparently exercise the rollup path with no fetch-call change. A query failure surfaces as an unavailable chart (em-dash / message), mirroring how the existing KPI tiles surface transient failures — it never breaks the hosting page.
### `KpiTrendChart.razor` (Central UI)
A reusable **custom inline-SVG** line/area chart — a polyline path with min/max + time-range axis labels, a responsive `viewBox`, and clean corporate styling. There is **no third-party charting library** (per the CLAUDE.md no-third-party-component-framework rule). The time window (e.g. 24h / 7d) is owned by the parent page.
A reusable **custom inline-SVG** line/area chart — a polyline path with min/max + time-range axis labels, a responsive `viewBox`, and clean corporate styling. There is **no third-party charting library** (per the CLAUDE.md no-third-party-component-framework rule). The time window is owned by the parent page; each of the four surfaces offers **24 h / 7 d / 30 d (720 h) / 90 d (2160 h)** toggle buttons — the two long windows are what exercise the rollup read path.
### Surfaces
@@ -143,16 +188,20 @@ Bound from the `ScadaBridge:KpiHistory` section on the central host (Options pat
| Option | Default | Notes |
|---|---|---|
| `SampleInterval` | `60s` | Recorder tick cadence. Must be `> 0`. |
| `RetentionDays` | `90` | Rows older than this are purged. Bounded to `[1, 3650]` days. |
| `PurgeInterval` | `1d` | Daily purge cadence. Must be `> 0`. |
| `RetentionDays` | `90` | Raw `KpiSample` rows older than this are purged. Bounded to `[1, 3650]` days. |
| `PurgeInterval` | `1d` | Daily purge cadence (drives **both** raw and rollup purges). Must be `> 0`. |
| `DefaultMaxSeriesPoints` | `200` | Default bucket cap for a series query when the caller does not override it. Bounded to `[2, 5000]`. |
| `RollupInterval` | `1h` | Hourly rollup fold cadence (`kpi-rollup` timer). Must be `> 0`. |
| `RollupLookbackHours` | `3` | Trailing whole hours each fold re-processes (idempotent self-heal for a missed tick). Bounded to `[1, 168]`. |
| `RollupRetentionDays` | `365` | `KpiRollupHourly` rows older than this are purged. Bounded to `[1, 3650]` days **and** required `>= RetentionDays`. |
| `RollupThresholdHours` | `168` | Raw-vs-rollup query-routing boundary (7 d); windows strictly wider read rollups. Minimum `24`. |
Validation fails fast at startup on a non-positive `SampleInterval` / `PurgeInterval` (which would stall the recorder / purge), an out-of-range `RetentionDays` (too short loses history; too long defeats retention), or an out-of-range `DefaultMaxSeriesPoints`.
Validation fails fast at startup on a non-positive `SampleInterval` / `PurgeInterval` / `RollupInterval` (which would stall the recorder / purge / fold), an out-of-range `RetentionDays` / `RollupLookbackHours` / `RollupRetentionDays` / `DefaultMaxSeriesPoints`, a `RollupThresholdHours` below 24 h, or a `RollupRetentionDays < RetentionDays` (which would leave long-range charts a hole where raw is purged but no rollup remains).
## Dependencies
- **Commons**: defines the `KpiSample` entity, the `IKpiSampleSource` and `IKpiHistoryRepository` interfaces, the `KpiSources` / `KpiScopes` catalogs, and the `KpiSeriesPoint` / `KpiSeriesBucketer` query types.
- **Configuration Database**: hosts the `KpiSample` table, its EF mapping, the migration, and the `KpiHistoryRepository` implementation.
- **Configuration Database**: hosts the `KpiSample` and `KpiRollupHourly` tables, their EF mappings, the migrations, and the `KpiHistoryRepository` implementation (raw + rollup fold / query / purge).
- **Cluster Infrastructure**: hosts the `kpi-history-recorder` cluster singleton with active/standby failover.
- **Host**: binds `KpiHistoryOptions`, registers the component on the central role, and starts the recorder singleton **outside** the readiness barrier.
- **Notification Outbox / Site Call Audit / Audit Log / Health Monitoring**: each contributes an `IKpiSampleSource` and the KPI/aggregator reads it reuses. KPI History depends on the `IKpiSampleSource` abstraction, not on these components directly.