dohertj2/scadalink-design
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: complete gRPC streaming channel — site host, docker config, docs, integration tests

Browse Source 
Switch site host to WebApplicationBuilder with Kestrel HTTP/2 gRPC server,
add GrpcPort/keepalive config, wire SiteStreamManager as ISiteStreamSubscriber,
expose gRPC ports in docker-compose, add site seed script, update all 10
requirement docs + CLAUDE.md + README.md for the new dual-transport architecture.
This commit is contained in:
Joseph Doherty
2026-03-21 12:38:33 -04:00
parent 3fe3c4161b
commit 416a03b782
34 changed files with 728 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/requirements/HighLevelReqs.md
View File

@@ -45,11 +45,13 @@
- **Machine Data Database**: A separate database for collected machine data (e.g., telemetry, measurements, events).
### 2.2 Communication: Central ↔ Site
- Central-to-site and site-to-central communication uses **Akka.NET ClusterClient/ClusterClientReceptionist** for cross-cluster messaging with automatic failover.
- **Site addressing**: Site Akka base addresses (NodeA and NodeB) are stored in the **Sites database table** and configured via the Central UI. Central creates a ClusterClient per site using both addresses as contact points (cached in memory, refreshed periodically and on admin changes) rather than relying on runtime registration messages from sites.
- Two transport layers are used for central-site communication:
- **Akka.NET ClusterClient/ClusterClientReceptionist**: Handles **command/control** messaging — deployments, instance lifecycle commands, subscribe/unsubscribe handshake, debug snapshots, health reports, remote queries, and integration routing. Provides automatic failover between contact points.
- **gRPC server-streaming (site→central)**: Handles **real-time data streaming** — attribute value updates and alarm state changes. Each site node hosts a **SiteStreamGrpcServer** on a dedicated HTTP/2 port (Kestrel, default port 8083). Central creates per-site **SiteStreamGrpcClient** instances to subscribe to site streams. gRPC provides HTTP/2 flow control and per-stream backpressure that ClusterClient lacks.
- **Site addressing**: Site Akka base addresses (NodeA and NodeB) and gRPC endpoints (GrpcNodeAAddress and GrpcNodeBAddress) are stored in the **Sites database table** and configured via the Central UI or CLI. Central creates a ClusterClient per site using both Akka addresses as contact points, and per-site gRPC clients using the gRPC addresses.
- **Central contact points**: Sites configure **multiple central contact points** (both central node addresses) for redundancy. ClusterClient handles failover between central nodes automatically.
- **Central as integration hub**: Central brokers requests between external systems and sites. For example, a recipe manager sends a recipe to central, which routes it to the appropriate site. MES requests machine values from central, which routes the request to the site and returns the response.
- **Real-time data streaming** is not continuous for all machine data. The only real-time stream is an **on-demand debug view** — an engineer in the central UI can open a live view of a specific instance's tag values and alarm states for troubleshooting purposes. This is session-based and temporary. The debug view subscribes to the site-wide Akka stream filtered by instance (see Section 8.1).
- **Real-time data streaming** is not continuous for all machine data. The only real-time stream is an **on-demand debug view** — an engineer in the central UI can open a live view of a specific instance's tag values and alarm states for troubleshooting purposes. This is session-based and temporary. The debug view subscribes via gRPC to the site's SiteStreamManager filtered by instance (see Section 8.1).
### 2.3 Site-Level Storage & Interface
- Sites have **no user interface** — they are headless collectors, forwarders, and script executors.
@@ -362,7 +364,7 @@ The central cluster hosts a **configuration and management UI** (no live machine
- **Database Connection Management**: Define named database connections for script use.
- **Inbound API Management**: Manage API keys (create, enable/disable, delete). Define API methods (name, parameters, return values, approved keys, implementation script). *(Admin role for keys, Design role for methods.)*
- **Instance Management**: Create instances from templates, bind data connections (per-attribute, with **bulk assignment** UI for selecting multiple attributes and assigning a data connection at once), set instance-level attribute overrides, assign instances to areas. **Disable** or **delete** instances.
- **Site & Data Connection Management**: Define sites (including optional NodeAAddress and NodeBAddress fields for Akka remoting paths), manage data connections and assign them to sites.
- **Site & Data Connection Management**: Define sites (including optional NodeAAddress and NodeBAddress fields for Akka remoting paths, and optional GrpcNodeAAddress and GrpcNodeBAddress fields for gRPC streaming endpoints), manage data connections and assign them to sites.
- **Area Management**: Define hierarchical area structures per site for organizing instances.
- **Deployment**: View diffs between deployed and current template-derived configurations, deploy updates to individual instances. Filter instances by area. Pre-deployment validation runs automatically before any deployment is sent.
- **System-Wide Artifact Deployment**: Explicitly deploy shared scripts, external system definitions, database connection definitions, data connection definitions, notification lists, and SMTP configuration to all sites or to an individual site (requires Deployment role). Per-site deployment is available via the Sites admin page.
@@ -373,7 +375,7 @@ The central cluster hosts a **configuration and management UI** (no live machine
- **Site Event Log Viewer**: Query and view operational event logs from site clusters (see Section 12).
### 8.1 Debug View
- **Subscribe-on-demand**: When an engineer opens a debug view for an instance, central subscribes to the **site-wide Akka stream** filtered by instance unique name. The site first provides a **snapshot** of all current attribute values and alarm states from the Instance Actor, then streams subsequent changes from the Akka stream.
- **Subscribe-on-demand**: When an engineer opens a debug view for an instance, central opens a **gRPC server-streaming subscription** to the site's `SiteStreamGrpcServer` for the instance, then requests a **snapshot** of all current attribute values and alarm states via ClusterClient. The gRPC stream delivers subsequent attribute value and alarm state changes directly from the site's `SiteStreamManager`.
- Attribute value stream messages are structured as: `[InstanceUniqueName].[AttributePath].[AttributeName]`, attribute value, attribute quality, attribute change timestamp.
- Alarm state stream messages are structured as: `[InstanceUniqueName].[AlarmName]`, alarm state (active/normal), priority, timestamp.
- The stream continues until the engineer **closes the debug view**, at which point central unsubscribes and the site stops streaming.