dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(kpi): trend charts state per-bucket-total semantics for rate metrics (plan R2-04 T7)

Browse Source
This commit is contained in:
Joseph Doherty
2026-07-13 10:12:58 -04:00
parent 9a1b160e79
commit 1c8b2bc745
2 changed files with 16 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/requirements/Component-KpiHistory.md
+4 -2
View File
@@ -148,15 +148,17 @@ Reads `ICentralHealthAggregator.GetAllSiteStates()` (in-memory, no DB). Scope: *
### Bucketed query
`IKpiHistoryRepository.GetRawSeriesAsync(source, metric, scope, scopeKey, fromUtc, toUtc, …)` returns the raw points for one series over `[fromUtc, toUtc]`. `GetHourlySeriesAsync(…)` returns the same ascending `KpiSeriesPoint` shape from `KpiRollupHourly` (projecting `HourStartUtc` → the point timestamp, `Value`), so the bucketer is agnostic to the source. `KpiSeriesBucketer.Bucket(series, fromUtc, toUtc, maxPoints)` then partitions the window into ≤ `maxPoints` time buckets and returns the **last value per bucket** as `KpiSeriesPoint(BucketStartUtc, Value)` — unchanged, and still applied on top of a rollup series (a 90 d rollup is ~2,160 points > the 200-point cap, so bucketing still thins it). The chart plots the single `Value` series; avg / min / max charting is still deferred, though the rollup's `MinValue` / `MaxValue` / `SampleCount` columns are now stored to unblock it.
`IKpiHistoryRepository.GetRawSeriesAsync(source, metric, scope, scopeKey, fromUtc, toUtc, …)` returns the raw points for one series over `[fromUtc, toUtc]`. `GetHourlySeriesAsync(…)` returns the same ascending `KpiSeriesPoint` shape from `KpiRollupHourly` (projecting `HourStartUtc` → the point timestamp, `Value`), so the bucketer is agnostic to the source. `KpiSeriesBucketer.Bucket(series, fromUtc, toUtc, maxPoints, aggregation)` then partitions the window into ≤ `maxPoints` time buckets and reduces each bucket **per-metric**: it keeps the **last value** for a **Gauge** series and **sums per bucket** for a **Rate** series, driven by the same `KpiMetricAggregationCatalog` the hourly fold consults. The previous unconditional last-value-per-bucket kept **1 of ~11 hourly sums** per 90 d bucket on the rollup path — the exact fold error the catalog's own doc warns about, re-introduced one layer up — and is fixed per arch-review 04 round 2, R3. The reduction is still applied on top of a rollup series (a 90 d rollup is ~2,160 points > the 200-point cap, so bucketing still thins it). The chart plots the single `Value` series; avg / min / max charting is still deferred, though the rollup's `MinValue` / `MaxValue` / `SampleCount` columns are now stored to unblock it.
### Raw-vs-rollup range routing
`GetRawSeriesAsync` and `GetHourlySeriesAsync` are selected by window width in the query service (below), governed by `RollupThresholdHours` (default 168 = 7 d). Windows **at or below** the threshold read raw samples (preserving intra-minute detail on the 24 h / 7 d charts); windows **strictly wider** than the threshold read the pre-aggregated hourly rollups (≈60× fewer rows scanned on the 30 d / 90 d charts). The boundary is strict — a window of exactly `RollupThresholdHours` stays on the raw path.
Rate charts keep **one unit (events per chart bucket)** on both sides of the boundary: the catalog-driven per-bucket sum reduction is applied to the raw-minute deltas and the hourly rollup sums alike, so widening 7 d → 30 d no longer changes a Rate chart's magnitude ~60×. Residual: an unbucketed just-over-threshold rollup series plots native per-hour sums vs. the at-threshold raw path's ~50-minute bucket sums (≤~1.2×) — accepted and documented, not hidden.
### Aggregation intent — gauge vs. rate
Not every metric folds the same way, and choosing wrong silently corrupts long-range totals. `KpiMetricAggregationCatalog.Resolve(source, metric)` classifies each `(Source, Metric)` pair as `KpiRollupAggregation.Gauge` (fold = **last-value-per-hour**) or `KpiRollupAggregation.Rate` (fold = **sum-per-hour**):
Not every metric folds the same way, and choosing wrong silently corrupts long-range totals. `KpiMetricAggregationCatalog.Resolve(source, metric)` classifies each `(Source, Metric)` pair as `KpiRollupAggregation.Gauge` (fold = **last-value-per-hour**) or `KpiRollupAggregation.Rate` (fold = **sum-per-hour**). The catalog now has **two consumers**: the hourly rollup fold (which picks last-value vs. sum per hour) and the chart-time bucket reduction in `KpiSeriesBucketer` (which picks last-value vs. sum per chart bucket) — one classification keeps both layers consistent:
- **Gauge** (last-value) — instantaneous readings where a per-hour sum is meaningless: `queueDepth`, `buffered`, `stuck`/`stuckCount`, `parked*`, `oldestPendingAgeSeconds`, `connectionsUp`/`connectionsDown`, `deployedInstances`, and every other metric not explicitly listed as a rate. This is the **safe default**: an unknown / newly-added metric resolves to Gauge and never crashes the fold or invents a total.
- **Rate** (sum-per-hour) — already-windowed per-interval deltas whose hour total is the sum of its minute samples: `NotificationOutbox/deliveredLastInterval`, `SiteCallAudit/failedLastInterval`, `SiteCallAudit/deliveredLastInterval`, and the per-report `SiteHealth` error counts `scriptErrors` / `alarmEvalErrors` / `deadLetters` / `eventLogWriteFailures`. Folding these as last-value would keep one minute's delta and discard the other 59, under-counting every long-range total.