docs(m5): design — audit hardening T3-T8 (T1 hash-chain + T2 Parquet stay deferred)
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,150 @@
|
|||||||
|
# M5 — Audit Hardening (T3–T8) — Design
|
||||||
|
|
||||||
|
**Status:** Approved (awaiting plan).
|
||||||
|
**Worktree/branch:** `worktree-m5-audit-hardening` off `main` (`e77e209`).
|
||||||
|
**Source:** Phase-2 milestone M5 from `docs/plans/2026-06-15-stillpending-completion-design.md`.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Harden the centralized Audit Log with six independent, ready-to-build items. Two
|
||||||
|
items originally listed under M5 — **T1 hash-chain tamper evidence** and **T2
|
||||||
|
Parquet export** — remain **deferred to v1.x** (per CLAUDE.md's audit design
|
||||||
|
decisions); their stubs (CLI `verify-chain` no-op, export `501`) stay unchanged.
|
||||||
|
|
||||||
|
## Scope (in)
|
||||||
|
|
||||||
|
T3 per-channel retention · T4 ParentExecutionId tag-cascade · T5 historical
|
||||||
|
backfill (reframed) · T6 per-node stuck KPIs · T7 structured response-capture
|
||||||
|
increments · T8 CLI `audit tree`.
|
||||||
|
|
||||||
|
## Scope (out / deferred to v1.x)
|
||||||
|
|
||||||
|
T1 hash-chain (no Hash/PrevHash columns, no real verify-chain), T2 Parquet
|
||||||
|
export (the `501` gate stays). Reversing those deferrals is a separate decision.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Items
|
||||||
|
|
||||||
|
### T8 — CLI `audit tree` (smallest; reuses existing server walk + UI)
|
||||||
|
The recursive execution-tree walk (`IAuditLogRepository.GetExecutionTreeAsync`,
|
||||||
|
backed by `IX_AuditLog_ParentExecution`) and the Blazor `ExecutionTreePage`
|
||||||
|
already exist; only an HTTP projection + CLI surface are missing.
|
||||||
|
- **Server:** add `GET /api/audit/tree?executionId=…` in
|
||||||
|
`AuditEndpoints.MapAuditAPI` → `repo.GetExecutionTreeAsync` → serialize
|
||||||
|
`ExecutionTreeNode[]`.
|
||||||
|
- **CLI:** add `audit tree --execution-id <guid> [--format table|json]` in
|
||||||
|
`AuditCommands` + an `AuditTreeHelpers` renderer (indented ASCII tree for
|
||||||
|
`table`; raw nodes for `json`), mirroring `AuditQueryHelpers`/`AuditExportHelpers`.
|
||||||
|
- No schema change. **Tests:** endpoint returns the tree; CLI renders a
|
||||||
|
multi-level tree + handles not-found.
|
||||||
|
|
||||||
|
### T6 — Per-node stuck-count KPIs
|
||||||
|
KPIs are per-site today; `SourceNode` is on the `Notification` and `SiteCalls`
|
||||||
|
rows but not aggregated.
|
||||||
|
- Add `ComputePerNodeKpisAsync` (group by `SourceNode`) parallel to the existing
|
||||||
|
`ComputePerSiteKpisAsync` in `NotificationOutboxRepository` and
|
||||||
|
`SiteCallAuditRepository`.
|
||||||
|
- New `PerNode…KpiRequest`/`Response` message pair per actor; register in each
|
||||||
|
actor's `Receive<>`.
|
||||||
|
- Surface a per-node breakdown on the existing KPI tiles
|
||||||
|
(`AuditKpiTiles`/`SiteCallKpiTiles`) — additive, behind the existing tiles.
|
||||||
|
- **Tests:** repository grouping returns correct per-node counts (stuck/parked/
|
||||||
|
queue-depth); message round-trip.
|
||||||
|
|
||||||
|
### T7 — Structured response-capture increments (no schema change)
|
||||||
|
- **(a) Inbound request headers** → captured into the existing `Extra` JSON in
|
||||||
|
`AuditWriteMiddleware.EmitInboundAudit`, passed through the existing header
|
||||||
|
redactor (auth headers redacted by default).
|
||||||
|
- **(b) `AuditInboundCeilingHits`** counter on `AuditCentralHealthSnapshot`
|
||||||
|
(alongside the existing failure counters), incremented when an inbound row
|
||||||
|
truncates (request or response hits `InboundMaxBytes`). Surfaced via the
|
||||||
|
health snapshot.
|
||||||
|
- **(c) Per-method opt-out** of body capture: a `SkipBodyCapture` flag on
|
||||||
|
`PerTargetRedactionOverride`, checked in the capture pipeline so a noisy/
|
||||||
|
sensitive method can suppress body capture (headers + metadata still recorded).
|
||||||
|
- **Tests:** request headers land in `Extra` and are redacted; ceiling-hit
|
||||||
|
increments the counter; opt-out suppresses body but keeps the row.
|
||||||
|
|
||||||
|
### T4 — `ParentExecutionId` tag-cascade (touches the actor model — high-risk)
|
||||||
|
Completes the execution tree beyond the inbound-API→routed-script case.
|
||||||
|
- **Alarm on-trigger:** thread a `Guid? parentExecutionId` through
|
||||||
|
`AlarmActor.SpawnAlarmExecutionActor` → `AlarmExecutionActor` →
|
||||||
|
`ScriptRuntimeContext`, so an alarm-triggered script chains to its firing
|
||||||
|
context (the alarm's own execution id where one exists; otherwise a root).
|
||||||
|
- **Nested `CallScript`/`CallShared`:** in `ScriptRuntimeContext`, pass **the
|
||||||
|
current run's `ExecutionId`** (not the inherited `_parentExecutionId`) as the
|
||||||
|
child invocation's `ParentExecutionId`, so `A → CallScript(B)` records B's
|
||||||
|
parent as A — a true multi-level tree.
|
||||||
|
- **Timer/expression-trigger top-level runs** stay roots (no spawner) — unchanged.
|
||||||
|
- **Tests:** alarm-triggered script row carries the expected parent; a 2-level
|
||||||
|
nested `CallScript` produces a chain A→B→C walkable by `GetExecutionTreeAsync`.
|
||||||
|
- **Risk:** serialized actor state + correlation plumbing; covered by targeted
|
||||||
|
SiteRuntime actor tests + a tree-walk integration assertion.
|
||||||
|
|
||||||
|
### T3 — Per-channel retention overrides (one design wrinkle, resolved)
|
||||||
|
Retention is a single global `RetentionDays`; the purge actor switches out whole
|
||||||
|
month partitions by `OccurredAtUtc` (channel-blind).
|
||||||
|
- Add `PerChannelRetentionDays` (`Dictionary<string,int>`, keyed by channel /
|
||||||
|
`Action` name) to `AuditLogOptions`, validated like the global value; a channel
|
||||||
|
override may only be **shorter** than the global window (longer is meaningless
|
||||||
|
under month-partition switch-out, which is governed by the largest retention).
|
||||||
|
- **Mechanism (resolved):** after the coarse global partition purge, the purge
|
||||||
|
actor runs a **bounded row-level delete** for channels whose override is
|
||||||
|
shorter than global (`DELETE … WHERE Action=@channel AND OccurredAtUtc<@thr`,
|
||||||
|
batched). This runs from the **purge/maintenance path, not the writer role** —
|
||||||
|
the append-only invariant binds the writer/ingest role, not maintenance. The
|
||||||
|
**M2.10 CI grep-guard is widened** to allow the purge actor's single audited
|
||||||
|
deletion call site (an allow-list entry, not a blanket exemption).
|
||||||
|
- **Tests:** a channel with a shorter override is purged earlier than the global;
|
||||||
|
channels without an override follow the global; the guard still rejects
|
||||||
|
UPDATE/DELETE everywhere except the sanctioned purge site.
|
||||||
|
|
||||||
|
### T5 — Historical backfill (reframed per the computed-column reality)
|
||||||
|
- **`SourceNode`** is a physical nullable column. For truly historical rows the
|
||||||
|
node-of-origin is **unknowable**, so the backfill sets a **configurable
|
||||||
|
sentinel** (default `"unknown"`) on `NULL` rows via a one-shot maintenance
|
||||||
|
command (run from the purge/maintenance path), rather than guessing a node.
|
||||||
|
- **`ExecutionId`/`ParentExecutionId`** are **persisted computed columns derived
|
||||||
|
from `DetailsJson`**; backfilling them means mutating the JSON, which
|
||||||
|
append-only forbids. These are **documented as a runbook limitation** (pre-feature
|
||||||
|
rows stay NULL) — no code.
|
||||||
|
- **Tests:** the SourceNode backfill sets the sentinel only on NULL rows within a
|
||||||
|
bounded range and is idempotent; documentation note added.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Cross-cutting
|
||||||
|
|
||||||
|
- **Shared seams:** `AuditLogOptions` (T3, T7), `AuditEndpoints.MapAuditAPI`
|
||||||
|
(T8), `AuditCommands` (T8), `AuditCentralHealthSnapshot` (T6, T7),
|
||||||
|
`IAuditLogRepository`/the KPI repositories (T6), the purge/maintenance role
|
||||||
|
(T3, T5). No AuditLog **schema** change in M5 (T1/T2 deferred).
|
||||||
|
- **Append-only:** the only new deletion is T3's purge-role channel delete +
|
||||||
|
T5's purge-role sentinel UPDATE — both maintenance-path, both reflected in the
|
||||||
|
CI guard's allow-list. Writer/ingest paths stay INSERT-only.
|
||||||
|
|
||||||
|
## Testing strategy
|
||||||
|
|
||||||
|
Per-item unit + targeted integration tests (above). T4 additionally gets a
|
||||||
|
tree-walk integration assertion. Full-solution build + targeted suites at the
|
||||||
|
integration step. No new infra dependency (Parquet deferred).
|
||||||
|
|
||||||
|
## Sequencing
|
||||||
|
|
||||||
|
Independent items, parallelizable by disjoint area:
|
||||||
|
- **Wave A (parallel):** T8 (CLI+endpoint), T6 (KPI repos+actors+tiles), T7
|
||||||
|
(middleware+health+redaction-override) — disjoint projects.
|
||||||
|
- **Wave B (parallel):** T4 (SiteRuntime actors — high-risk), T3 (AuditLog
|
||||||
|
options+purge actor+CI guard), T5 (purge-path backfill command + runbook).
|
||||||
|
- **Wave C:** integration verification + docs (Component-AuditLog/-CLI, CLAUDE.md
|
||||||
|
KPI/retention notes, runbook).
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- **T4** actor-model correlation (serialized state) — targeted tests + tree-walk
|
||||||
|
assertion.
|
||||||
|
- **T3** append-only tension — resolved via maintenance-role delete + CI-guard
|
||||||
|
allow-list; verify the guard still blocks all other DELETE/UPDATE.
|
||||||
|
- **T5** node-of-origin unknowable — sentinel + documented limitation (no false
|
||||||
|
precision).
|
||||||
Reference in New Issue
Block a user