9b5cb3dd9d
Every AttributeValueChanged/AlarmStateChanged rode its own gRPC message on
SiteStreamService; at target scale that is ~37.5k messages/s/site of pure
framing overhead. Coalesce them, additively, with no new RPC.
Wire (sitestream.proto, regenerated via docker/regen-proto.sh sitestream):
InstanceStreamRequest.batching_supported = 3
SiteStreamRequest.batching_supported = 2
SiteStreamEvent.batch = 4 (new oneof case)
SiteStreamEventBatch { repeated SiteStreamEvent events = 1 }
The proto3 default of batching_supported IS the negotiation, and it is
load-bearing: a batch frame reaches a pre-R2 central as EventOneofCase.None,
whose ConvertToDomainEvent returns null — the whole batch would vanish with
no error anywhere. An old central cannot set the flag so it never receives
one; an old site ignores the unknown request field and keeps sending
per-event frames, which the new client's ForEachEvent handles as the
single-event case. Both skew directions are covered by tests that go through
a real proto serialize/parse round-trip.
Server: SiteStreamEventBatcher, a per-subscriber pump replacing the handler's
await-foreach/WriteAsync loop and byte-identical to it at a cap of 1. It
never delays a lone event — it drains the already-queued backlog for free and
lingers only once a backlog is proven — emits a single-event buffer as a
plain frame, preserves order and per-event Timestamps exactly, and flushes
what is buffered when the send channel's writer completes. It sits DOWNSTREAM
of StreamRelayActor's bounded DropOldest channel, so it changes framing only
and does not move the burst ceiling (deferred register row 31, which lives in
the shared publish stage upstream of the BroadcastHub).
Client: sets the flag on both subscriptions and unpacks in order into the
existing per-event pipeline, so SiteAlarmAggregatorActor,
DebugStreamBridgeActor, consumer-keepalive/orphan logic,
reconnect-on-graceful-completion, generation fencing, the (siteId, endpoint)
factory key and IsLive semantics are untouched.
Options (validated): GrpcStreamBatchMaxEvents 100 (1 disables),
GrpcStreamBatchWindow 25 ms — validated strictly under 250 ms, the load
test's end-to-end P99 threshold. Measured worst case (trickle-with-backlog,
window-bound rather than cap-bound): P50 13.8 ms, P99 25.4 ms, max 25.8 ms;
cap-bound case sent 600 queued events in 6 frames.
Telemetry: histogram scadabridge.site.stream.batch_size tagged by stream
kind, recorded only on negotiated streams (per-event otherwise). It rides
ScadaBridgeTelemetry.MeterName, already in the ObservedMeters allowlist.
Docs: Component-Communication.md gains an Event Batching section; CLAUDE.md's
gRPC streaming bullet records the wire shape and the negotiation rationale.
245 lines
11 KiB
Protocol Buffer
245 lines
11 KiB
Protocol Buffer
syntax = "proto3";
|
|
option csharp_namespace = "ZB.MOM.WW.ScadaBridge.Communication.Grpc";
|
|
package sitestream;
|
|
|
|
import "google/protobuf/timestamp.proto";
|
|
import "google/protobuf/wrappers.proto"; // Int32Value
|
|
|
|
service SiteStreamService {
|
|
rpc SubscribeInstance(InstanceStreamRequest) returns (stream SiteStreamEvent);
|
|
// Site-wide, alarm-only live stream (aggregated Alarm Summary): every
|
|
// AlarmStateChanged for ALL instances on the site, no per-instance filter.
|
|
// Attribute updates are never carried on this stream.
|
|
rpc SubscribeSite(SiteStreamRequest) returns (stream SiteStreamEvent);
|
|
rpc IngestAuditEvents(AuditEventBatch) returns (IngestAck);
|
|
rpc IngestCachedTelemetry(CachedTelemetryBatch) returns (IngestAck);
|
|
rpc PullAuditEvents(PullAuditEventsRequest) returns (PullAuditEventsResponse);
|
|
rpc PullSiteCalls(PullSiteCallsRequest) returns (PullSiteCallsResponse);
|
|
}
|
|
|
|
message InstanceStreamRequest {
|
|
string correlation_id = 1;
|
|
string instance_unique_name = 2;
|
|
// Client-declared BATCH NEGOTIATION (R2, event batching). When true the client
|
|
// understands the SiteStreamEventBatch oneof case and the server may coalesce
|
|
// consecutive events into one frame. proto3 defaults this to false, so an OLD
|
|
// central that never sets it keeps receiving one frame per event — that default
|
|
// IS the negotiation, and it is what makes new-site↔old-central safe. A NEW
|
|
// central sets it against an OLD site, which ignores the unknown field and
|
|
// keeps sending per-event frames the new client also accepts. Additive-only.
|
|
bool batching_supported = 3;
|
|
}
|
|
|
|
// Request for the site-wide, alarm-only SubscribeSite stream. Unlike
|
|
// InstanceStreamRequest there is NO instance filter — the stream carries alarm
|
|
// transitions for every instance on the site.
|
|
message SiteStreamRequest {
|
|
string correlation_id = 1;
|
|
// See InstanceStreamRequest.batching_supported. Additive-only.
|
|
bool batching_supported = 2;
|
|
}
|
|
|
|
message SiteStreamEvent {
|
|
string correlation_id = 1;
|
|
oneof event {
|
|
AttributeValueUpdate attribute_changed = 2;
|
|
AlarmStateUpdate alarm_changed = 3;
|
|
// Coalesced frame (R2). Emitted ONLY when the subscription request set
|
|
// batching_supported = true. A batch is never nested inside a batch, and a
|
|
// single event is always sent as a plain attribute_changed/alarm_changed
|
|
// frame — so a quiet stream's wire shape is byte-identical to before.
|
|
SiteStreamEventBatch batch = 4;
|
|
}
|
|
}
|
|
|
|
// Coalesced carrier for several consecutive stream events (R2). Ordering is
|
|
// significant: events appear in the exact order the site produced them, and the
|
|
// client unpacks them in order into the same per-event pipeline, so per-event
|
|
// Timestamp fidelity and downstream sequencing are unchanged.
|
|
//
|
|
// The inner events deliberately leave correlation_id EMPTY — the enclosing
|
|
// SiteStreamEvent carries it once for the whole frame, which is the byte saving
|
|
// batching exists for. No consumer reads the inner correlation_id.
|
|
message SiteStreamEventBatch {
|
|
repeated SiteStreamEvent events = 1;
|
|
}
|
|
|
|
enum Quality {
|
|
QUALITY_UNSPECIFIED = 0;
|
|
QUALITY_GOOD = 1;
|
|
QUALITY_UNCERTAIN = 2;
|
|
QUALITY_BAD = 3;
|
|
}
|
|
|
|
enum AlarmStateEnum {
|
|
ALARM_STATE_UNSPECIFIED = 0;
|
|
ALARM_STATE_NORMAL = 1;
|
|
ALARM_STATE_ACTIVE = 2;
|
|
}
|
|
|
|
// Severity level for an active alarm. Binary trigger types (ValueMatch,
|
|
// RangeViolation, RateOfChange) always emit ALARM_LEVEL_NONE. The HiLo
|
|
// trigger type emits one of the directional values.
|
|
enum AlarmLevelEnum {
|
|
ALARM_LEVEL_NONE = 0;
|
|
ALARM_LEVEL_LOW = 1;
|
|
ALARM_LEVEL_LOW_LOW = 2;
|
|
ALARM_LEVEL_HIGH = 3;
|
|
ALARM_LEVEL_HIGH_HIGH = 4;
|
|
}
|
|
|
|
message AttributeValueUpdate {
|
|
string instance_unique_name = 1;
|
|
string attribute_path = 2;
|
|
string attribute_name = 3;
|
|
string value = 4;
|
|
Quality quality = 5;
|
|
google.protobuf.Timestamp timestamp = 6;
|
|
}
|
|
|
|
message AlarmStateUpdate {
|
|
string instance_unique_name = 1;
|
|
string alarm_name = 2;
|
|
AlarmStateEnum state = 3;
|
|
int32 priority = 4;
|
|
google.protobuf.Timestamp timestamp = 5;
|
|
AlarmLevelEnum level = 6; // ALARM_LEVEL_NONE for binary trigger types; set by HiLo.
|
|
string message = 7; // Optional per-band operator message; empty when unset.
|
|
|
|
// Native alarm enrichment (additive — computed alarms leave these at defaults).
|
|
// kind: "Computed" | "NativeOpcUa" | "NativeMxAccess".
|
|
string kind = 8;
|
|
bool active = 9; // unified condition: active vs inactive
|
|
bool acknowledged = 10; // acked vs unacked
|
|
bool confirmed = 11; // confirmed (false when not confirmable)
|
|
string shelve_state = 12; // Unshelved | OneShotShelved | TimedShelved | PermanentShelved
|
|
bool suppressed = 13;
|
|
string source_reference = 14; // native per-condition key; empty for computed
|
|
string alarm_type_name = 15;
|
|
string category = 16;
|
|
string operator_user = 17;
|
|
string operator_comment = 18;
|
|
google.protobuf.Timestamp original_raise_time = 19; // null when unknown
|
|
string current_value = 20;
|
|
string limit_value = 21;
|
|
string native_source_canonical_name = 22; // native binding canonical name; empty for computed
|
|
bool is_configured_placeholder = 23; // true for a quiet-binding placeholder row
|
|
|
|
// Ack timestamp for the condition; null while unacknowledged and on computed alarms
|
|
// (MES alarm-status API §6.4). The site stamps the SOURCE's own ack instant where the
|
|
// protocol supplies one (OPC UA A&C AckedState/TransitionTime) and the DCL's observation
|
|
// time of the ack transition where it does not (MxAccess Gateway).
|
|
google.protobuf.Timestamp ack_time = 24;
|
|
}
|
|
|
|
// Audit Log (#23) telemetry: single lifecycle event ferried from a site SQLite
|
|
// hot-path row to central via IngestAuditEvents. Mirrors AuditEvent (Commons)
|
|
// minus the site-local ForwardState and the central IngestedAtUtc (set on ingest).
|
|
message AuditEventDto {
|
|
string event_id = 1;
|
|
google.protobuf.Timestamp occurred_at_utc = 2;
|
|
string channel = 3;
|
|
string kind = 4;
|
|
string correlation_id = 5; // empty string represents null
|
|
string source_site_id = 6;
|
|
string source_instance_id = 7;
|
|
string source_script = 8;
|
|
string actor = 9;
|
|
string target = 10;
|
|
string status = 11;
|
|
google.protobuf.Int32Value http_status = 12; // null when absent
|
|
google.protobuf.Int32Value duration_ms = 13;
|
|
string error_message = 14;
|
|
string error_detail = 15;
|
|
string request_summary = 16;
|
|
string response_summary = 17;
|
|
bool payload_truncated = 18;
|
|
string extra = 19;
|
|
string execution_id = 20; // empty string represents null
|
|
string parent_execution_id = 21; // empty string represents null
|
|
string source_node = 22; // empty string represents null
|
|
}
|
|
|
|
message AuditEventBatch { repeated AuditEventDto events = 1; }
|
|
message IngestAck { repeated string accepted_event_ids = 1; }
|
|
|
|
// Audit Log (#23) M3 cached-call combined telemetry: a single packet carries
|
|
// both the AuditEvent row to insert and the SiteCalls operational-state upsert
|
|
// for one lifecycle event of a cached outbound call. Central writes both rows
|
|
// in one MS SQL transaction so the audit and operational mirrors never drift.
|
|
message SiteCallOperationalDto {
|
|
string tracked_operation_id = 1; // GUID string ("D" format)
|
|
string channel = 2; // "ApiOutbound" | "DbOutbound"
|
|
string target = 3;
|
|
string source_site = 4;
|
|
string status = 5; // AuditStatus name
|
|
int32 retry_count = 6;
|
|
string last_error = 7; // empty when null
|
|
google.protobuf.Int32Value http_status = 8;
|
|
google.protobuf.Timestamp created_at_utc = 9;
|
|
google.protobuf.Timestamp updated_at_utc = 10;
|
|
google.protobuf.Timestamp terminal_at_utc = 11; // absent when not terminal
|
|
string source_node = 12; // empty string represents null
|
|
}
|
|
|
|
message CachedTelemetryPacket {
|
|
AuditEventDto audit_event = 1;
|
|
SiteCallOperationalDto operational = 2;
|
|
}
|
|
|
|
message CachedTelemetryBatch { repeated CachedTelemetryPacket packets = 1; }
|
|
|
|
// Audit Log (#23) M6 reconciliation pull: central→site request for any
|
|
// site-local AuditLog rows with OccurredAtUtc >= since_utc that have not yet
|
|
// been ingested centrally (ForwardState in {Pending, Forwarded}). Rows are NOT
|
|
// flipped to Reconciled when they are served — only when a LATER pull's cursor
|
|
// proves central consumed them (see after_id), so a fault between the response
|
|
// leaving the site and central committing it re-serves the rows instead of
|
|
// silently losing them (at-least-once).
|
|
// more_available signals batch_size was saturated so the caller knows to
|
|
// issue a follow-up pull with an advanced since_utc cursor.
|
|
message PullAuditEventsRequest {
|
|
google.protobuf.Timestamp since_utc = 1;
|
|
int32 batch_size = 2;
|
|
// Composite-keyset cursor (WP2.3), mirroring PullSiteCallsRequest.after_id:
|
|
// the EventId ("D" GUID form) of the last row central has already CONSUMED at
|
|
// since_utc. When set, the site returns only rows strictly after the composite
|
|
// (OccurredAtUtc, EventId) pair — un-pinning a batch that would otherwise stall
|
|
// when more than batch_size rows share one since_utc instant — AND treats the
|
|
// cursor as proof of receipt: everything at or before it is flipped to
|
|
// Reconciled. Empty (the proto3 string default) preserves the legacy inclusive
|
|
// >= behaviour, under which only rows strictly older than since_utc are proven
|
|
// received. Additive-only.
|
|
string after_id = 3;
|
|
}
|
|
|
|
message PullAuditEventsResponse {
|
|
repeated AuditEventDto events = 1;
|
|
bool more_available = 2;
|
|
}
|
|
|
|
// Site Call Audit (#22) reconciliation pull: central→site request for any
|
|
// site-local operation-tracking rows whose UpdatedAtUtc >= since_utc — the
|
|
// self-heal feed that backfills the eventually-consistent central SiteCalls
|
|
// mirror when best-effort push telemetry is lost. Mirrors PullAuditEvents
|
|
// but is a SEPARATE RPC (the tracking store is the operational source of
|
|
// truth, distinct from the site audit queue). more_available signals
|
|
// batch_size was saturated so the caller advances since_utc and pulls again.
|
|
message PullSiteCallsRequest {
|
|
google.protobuf.Timestamp since_utc = 1;
|
|
int32 batch_size = 2;
|
|
// Composite-keyset cursor (Task 15): the TrackedOperationId ("D" GUID form) of
|
|
// the last row already consumed at since_utc. When set, the site returns only
|
|
// rows strictly after (since_utc, after_id) under a deterministic
|
|
// (UpdatedAtUtc, TrackedOperationId) order — un-pinning a batch that would
|
|
// otherwise stall when more than batch_size rows share one since_utc instant.
|
|
// Empty (the proto3 string default) preserves the legacy inclusive >= behaviour,
|
|
// so an older central that never sets it is unaffected. Additive-only.
|
|
string after_id = 3;
|
|
}
|
|
|
|
message PullSiteCallsResponse {
|
|
repeated SiteCallOperationalDto operationals = 1;
|
|
bool more_available = 2;
|
|
}
|