ScadaBridge/docs/operations/metrics-reference.md

Metrics instrument reference

Every application metric ScadaBridge emits, derived from the code (Gitea #99). Source of truth: src/ZB.MOM.WW.ScadaBridge.Commons/Observability/ScadaBridgeTelemetry.cs plus src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SiteAuditTelemetryMetrics.cs. To regenerate this list:

grep -rnE "Create(Counter|Histogram|UpDownCounter|Observable[A-Za-z]+)<" src --include='*.cs'

Meter

All instruments below live on one meter, ZB.MOM.WW.ScadaBridge (ScadaBridgeTelemetry.MeterName). The meter is exported only because it is in the ZbTelemetryOptions.Meters allowlist (SiteServiceRegistration.ObservedMeters); a new meter that is not added there is silently never scraped. That list also carries the LocalDb replication meter (LocalDbMetrics.MeterName), which is owned by the ZB.MOM.WW.LocalDb library and not catalogued here.

Instrument names are dotted (scadabridge.site.script.runs); Prometheus exposes them with underscores (scadabridge_site_script_runs_total for counters, plus unit suffixes). Instruments are no-ops until a listener attaches. Tag values listed are the ones emitted by the code today.

Counters

Instrument	Unit	Tags	Meaning / alert guidance
scadabridge.deployments.applied	1	none	Deployments successfully applied. Pair with deployments.failed for the success ratio.
scadabridge.deployments.failed	1	reason = timeout | error | site_failed	Deployments that ended in terminal failure. Alert on any sustained rate.
scadabridge.store_and_forward.activity	1	action (Queued, Delivered, Retried, Parked, Retry, Discard), category	S&F lifecycle events. A sustained Parked rate means messages are exhausting retries (the queue-depth gauge cannot tell drain from park).
scadabridge.site.script.runs	1	outcome = completed | failed | timed_out	Finished site script runs. Deliberately not tagged by instance/script (unbounded cardinality).
scadabridge.site.data_connections.transitions	1	state (new ConnectionHealth)	Real data-connection state changes only (not repeats). Shows flapping the gauges average away.
scadabridge.inbound_api.requests	1	method	Inbound API requests per API method.
scadabridge.inbound_api.abandoned_executions	1	method	Inbound executions abandoned after timeout/abort while the handler still ran.
scadabridge.store_and_forward.replication.failures	1	none	S&F buffer replication ops that failed to reach the peer node.
scadabridge.store_and_forward.resync.completed	1	none	Anti-entropy resyncs the standby acknowledged as applied.
scadabridge.store_and_forward.resync.ack_missing	1	none	Resyncs answered by the active node but not acknowledged by the standby in the ack window (lost chunks / dead peer).
scadabridge.site.alarm_cache.reconnects	1	none	Live-alarm aggregator site-wide gRPC stream reconnects. A sustained climb = flapping site link.
scadabridge.site.alarm_cache.buffer_dropped	1	none	Live deltas evicted from an aggregator's bounded pre-seed buffer (drop-oldest).
scadabridge.site.script_compile_cache.evictions	1	none	Entries evicted from the script compile cache at its bound (#68). A sustained climb = cache thrash and working-set growth; raise ScadaBridge:SiteRuntime:ScriptCompileCacheMaxEntries.
scadabridge.central.debug_view.presnapshot_dropped	1	none	Debug events evicted from a central debug session's pre-snapshot buffer.
scadabridge.site.stream.events_dropped	1	stream = instance | site-alarms	Events evicted from a site gRPC stream's bounded send channel.
scadabridge.central.transport.import_sessions.rejected	1	none	Bundle-import sessions refused because MaxConcurrentImportSessions was exhausted. Rising while the open gauge sits at the cap = session leak; rising while it oscillates below = real concurrency (raise the cap).
scadabridge.site.event_log.purge_ceiling_hits	1	none	Event-log purge ticks that stopped at MaxPurgedRowsPerTick with work remaining. A burst that stops is a converging backlog; hits that never stop mean the ceiling/interval is mis-sized.
scadabridge.site.audit.rejected_rows	1	none	Site audit rows central permanently refused MaxRejectedPushAttempts times; moved to Rejected (still served by the reconciliation pull).

Histograms

Instrument	Unit	Tags	Meaning
scadabridge.site.stream.batch_size	1 (events)	stream	Events per site gRPC stream frame; recorded only for subscriptions that negotiated batching. Pinned at 1 = window never sees a backlog.
scadabridge.site.script.duration	ms	outcome = completed | failed | timed_out	Site script run duration.

Observable gauges

Instrument	Unit	Tags	Meaning / alert guidance
scadabridge.site.stream.subscribers.active	1	none	Open central-to-site gRPC subscriber streams (Debug View + site-wide alarm feed). Not data connections; 0 on a healthy unwatched site. (Renamed from scadabridge.site.connection.up.)
scadabridge.site.data_connections.connected	1	none	Data connections on this node currently Connected.
scadabridge.site.data_connections.total	1	none	Data connections known to this node in any state. Alert on connected < total on the active node; the standby legitimately reports 0, so alert on the active node or max() across the pair.
scadabridge.store_and_forward.queue.depth	items	none	Current S&F queue depth.
scadabridge.site.alarm_cache.aggregators.active	1	none	Running per-site live-alarm aggregators on the active central node.
scadabridge.central.transport.import_sessions.open	1	none	Open bundle-import sessions on this central node (each pins a decrypted bundle in memory). 0 on sites. A plateau at the cap indicates leaked sessions.

No UpDownCounters or observable counters are defined.