AuditLog chain-head export: failures are log-only, no health counter #115

Closed
opened 2026-11-09 23:32:37 -05:00 by dohertj2 · 1 comment
Owner

Follow-up from #107 item 1 (c4d7ec9b). AuditChainHeadExportService logs an export failure (share offline, syslog collector unreachable) at Error and retries the same heads next tick, but nothing surfaces on the central health dashboard — the purge path has PurgeFailures, the head export has no counter. A sink that stays down silently stops the external anchor.

Proposal: an ExportFailures counter (or last-success timestamp) on AuditCentralHealthSnapshot, and consider a KPI for "months whose current head has not been exported for > 2 intervals". Also note: UDP syslog is fire-and-forget, so a dropped datagram is invisible by design — TCP is the recommended transport where that matters (documented).

Follow-up from #107 item 1 (`c4d7ec9b`). `AuditChainHeadExportService` logs an export failure (share offline, syslog collector unreachable) at Error and retries the same heads next tick, but nothing surfaces on the central health dashboard — the purge path has `PurgeFailures`, the head export has no counter. A sink that stays down silently stops the external anchor. Proposal: an `ExportFailures` counter (or last-success timestamp) on `AuditCentralHealthSnapshot`, and consider a KPI for "months whose current head has not been exported for > 2 intervals". Also note: UDP syslog is fire-and-forget, so a dropped datagram is invisible by design — TCP is the recommended transport where that matters (documented).
dohertj2 added the type:followuppriority:p3 labels 2026-11-09 23:32:37 -05:00
Author
Owner

Fixed in 533414a1 (main). Head-export tick failures now increment counter scadabridge.audit.chain_head_export.failures (shared ScadaBridge meter) and update AuditChainHeadExportStatus; new /health/ready entry audit-chain-head-export (Host AuditChainHeadExportHealthCheck) is Degraded (HTTP 200) while the last tick failed or there has been no success for > 2 intervals, Healthy when disabled. UDP syslog fire-and-forget caveat + TCP recommendation documented. Evidence: AuditChainHeadExportServiceTests.FailedTick_RecordsMetricAndStatus_SuccessResetsConsecutive (MeterListener) and AuditChainHeadExportHealthCheckTests; mutation-proved (removing RecordFailure fails it). Not done: a months-not-exported KPI on AuditCentralHealthSnapshot (the staleness check covers the silent-anchor case).

Fixed in 533414a1 (main). Head-export tick failures now increment counter scadabridge.audit.chain_head_export.failures (shared ScadaBridge meter) and update AuditChainHeadExportStatus; new /health/ready entry audit-chain-head-export (Host AuditChainHeadExportHealthCheck) is Degraded (HTTP 200) while the last tick failed or there has been no success for > 2 intervals, Healthy when disabled. UDP syslog fire-and-forget caveat + TCP recommendation documented. Evidence: AuditChainHeadExportServiceTests.FailedTick_RecordsMetricAndStatus_SuccessResetsConsecutive (MeterListener) and AuditChainHeadExportHealthCheckTests; mutation-proved (removing RecordFailure fails it). Not done: a months-not-exported KPI on AuditCentralHealthSnapshot (the staleness check covers the silent-anchor case).
Sign in to join this conversation.