Resolve all high-severity architecture review findings

2026-09-09 16:13:58 -04:00
parent 8bfe65512e
commit 146a13238f
192 changed files with 3160 additions and 753 deletions
@@ -452,3 +452,33 @@ Disconnect is detected at the **transport layer**, never via an application-leve
 - **Audit Log (#23)**: Sites forward audit-event telemetry (push) and serve the `PullAuditEvents` gRPC reconciliation pull from their `ISiteAuditQueue`; the central `AuditLogIngestActor` is the ingest target for both the push path and the combined cached-call telemetry packet.
 - **Site Event Logging**: Event log queries are routed through communication.
 - **Management Service**: The ManagementActor runs at `/user/management` on central nodes and is reached **in-process** via `ManagementActorHolder`. The CLI communicates with it over the central HTTP `/management` endpoints (HTTP Basic), not over any cluster transport. It is no longer registered with ClusterClientReceptionist (that registration, and the whole ClusterClient surface, is gone).
+
+### Control-plane ownership, retry safety and message limits (2026-09-09)
+
+Both gRPC servers and every application client channel explicitly set maximum send and receive
+message sizes to **32 MiB** through `GrpcMessageLimits`. Audit push and reconciliation normally
+use **2 MiB** batches measured from the complete protobuf message fields, repeated-message
+length/tag prefixes and gRPC framing. The shared limits must move together on both peers.
+
+On the central control service, the PSK interceptor stores a verified site identity in the
+server call context. Before dispatch it compares that identity to notification source site,
+reconciliation site identifier, health report and heartbeat site, **each** audit row's source
+site and **both** halves of every cached telemetry packet. A mismatch or missing owner denies
+the request with `PermissionDenied` before any handler or partial batch runs. Execution and
+parent-execution linkage may cross sites; those identifiers are not ownership fields.
+Notification status queries carry the verified identity to the outbox, which returns no result
+for a notification stored under another site. An unrecognized protobuf request is denied until
+its ownership rule is explicitly implemented.
+
+`Unavailable` alone does not prove a command was unsent: it can represent a connection loss
+after a device write, deployment or script call ran. The site transport automatically retries
+only an explicit read-only command whitelist, or a server readiness rejection carrying the
+`scadabridge-dispatch: not-started` trailer emitted before actor dispatch. An ambiguous mutation
+returns `Aborted` with `OUTCOME_UNKNOWN`; the caller must inspect the operation before retrying.
+The provider selects the peer for subsequent independent calls, without replaying the failed
+mutation. A secondary attempt that loses its response has the same uncertain-outcome behavior.
+
+Site-to-central notification submission and audit ingestion can retry because their durable
+identifiers are idempotent; status reads and sequence-guarded health reports can retry as well.
+Reconciliation mints fetch tokens and only retries a proven pre-dispatch rejection. Attempts
+share the original absolute deadline. No transport retries `DeadlineExceeded` across nodes.