feat: complete gRPC streaming channel — site host, docker config, docs, integration tests

Switch site host to WebApplicationBuilder with Kestrel HTTP/2 gRPC server, add GrpcPort/keepalive config, wire SiteStreamManager as ISiteStreamSubscriber, expose gRPC ports in docker-compose, add site seed script, update all 10 requirement docs + CLAUDE.md + README.md for the new dual-transport architecture.
2026-03-21 12:38:33 -04:00
parent 3fe3c4161b
commit 416a03b782
34 changed files with 728 additions and 156 deletions
--- a/docker/README.md
+++ b/docker/README.md
@@ -29,7 +29,8 @@ Local Docker deployment of the full ScadaLink cluster topology: a 2-node central
 │  (Test Plant A)    │ │  (Test Plant B)    │ │  (Test Plant C)    │
 │                    │ │                    │ │                    │
 │  node-a ◄──► node-b│ │  node-a ◄──► node-b│ │  node-a ◄──► node-b│
-│  :9021      :9022  │ │  :9031      :9032  │ │  :9041      :9042  │
+│  Akka :9021 :9022  │ │  Akka :9031 :9032  │ │  Akka :9041 :9042  │
+│  gRPC :9023 :9024  │ │  gRPC :9033 :9034  │ │  gRPC :9043 :9044  │
 └────────────────────┘ └────────────────────┘ └────────────────────┘
 ```

@@ -39,7 +40,7 @@ Runs the web UI (Blazor Server), Template Engine, Deployment Manager, Security,

 ### Site Clusters (active/standby each)

-Each site cluster runs Site Runtime, Data Connection Layer, Store-and-Forward, and Site Event Logging. Sites connect to OPC UA for device data and to the central cluster via Akka.NET remoting. Deployed configurations and S&F buffers are stored in local SQLite databases per node.
+Each site cluster runs Site Runtime, Data Connection Layer, Store-and-Forward, and Site Event Logging. Sites connect to OPC UA for device data and to the central cluster via Akka.NET remoting. Each site node also hosts a gRPC streaming server (port 8083) that central nodes connect to for real-time attribute value and alarm state streams. Deployed configurations and S&F buffers are stored in local SQLite databases per node.

 | Site Cluster | Site Identifier | Central UI Name |
 |-------------|-----------------|-----------------|
@@ -51,19 +52,19 @@ Each site cluster runs Site Runtime, Data Connection Layer, Store-and-Forward, a

 ### Application Nodes

-| Node | Container Name | Host Web Port | Host Akka Port | Internal Ports |
-|------|---------------|---------------|----------------|----------------|
-| Traefik LB | `scadalink-traefik` | 9000 | — | 80 (proxy), 8080 (dashboard) |
-| Central A | `scadalink-central-a` | 9001 | 9011 | 5000 (web), 8081 (Akka) |
-| Central B | `scadalink-central-b` | 9002 | 9012 | 5000 (web), 8081 (Akka) |
-| Site-A A | `scadalink-site-a-a` | — | 9021 | 8082 (Akka) |
-| Site-A B | `scadalink-site-a-b` | — | 9022 | 8082 (Akka) |
-| Site-B A | `scadalink-site-b-a` | — | 9031 | 8082 (Akka) |
-| Site-B B | `scadalink-site-b-b` | — | 9032 | 8082 (Akka) |
-| Site-C A | `scadalink-site-c-a` | — | 9041 | 8082 (Akka) |
-| Site-C B | `scadalink-site-c-b` | — | 9042 | 8082 (Akka) |
+| Node | Container Name | Host Web Port | Host Akka Port | Host gRPC Port | Internal Ports |
+|------|---------------|---------------|----------------|----------------|----------------|
+| Traefik LB | `scadalink-traefik` | 9000 | — | — | 80 (proxy), 8080 (dashboard) |
+| Central A | `scadalink-central-a` | 9001 | 9011 | — | 5000 (web), 8081 (Akka) |
+| Central B | `scadalink-central-b` | 9002 | 9012 | — | 5000 (web), 8081 (Akka) |
+| Site-A A | `scadalink-site-a-a` | — | 9021 | 9023 | 8082 (Akka), 8083 (gRPC) |
+| Site-A B | `scadalink-site-a-b` | — | 9022 | 9024 | 8082 (Akka), 8083 (gRPC) |
+| Site-B A | `scadalink-site-b-a` | — | 9031 | 9033 | 8082 (Akka), 8083 (gRPC) |
+| Site-B B | `scadalink-site-b-b` | — | 9032 | 9034 | 8082 (Akka), 8083 (gRPC) |
+| Site-C A | `scadalink-site-c-a` | — | 9041 | 9043 | 8082 (Akka), 8083 (gRPC) |
+| Site-C B | `scadalink-site-c-b` | — | 9042 | 9044 | 8082 (Akka), 8083 (gRPC) |

-Port block pattern: `90X1`/`90X2` where X = 0 (central), 1 (web), 2 (site-a), 3 (site-b), 4 (site-c).
+Port block pattern: `90X1`/`90X2` (Akka), `90X3`/`90X4` (gRPC) where X = 0 (central), 2 (site-a), 3 (site-b), 4 (site-c). gRPC streaming ports are used by central nodes to subscribe to real-time site data streams.

 ### Infrastructure Services (from `infra/docker-compose.yml`)

@@ -85,6 +86,7 @@ docker/
 ├── docker-compose.yml                  # 8-node application stack
 ├── build.sh                            # Build Docker image
 ├── deploy.sh                           # Build + deploy all containers
+├── seed-sites.sh                       # Create test sites with Akka + gRPC addresses
 ├── teardown.sh                         # Stop and remove containers
 ├── central-node-a/
 │   ├── appsettings.Central.json        # Central node A configuration
@@ -130,6 +132,9 @@ cd infra && docker compose up -d && cd ..

 # 2. Build and deploy all 8 ScadaLink nodes
 docker/deploy.sh
+
+# 3. Seed test sites (first-time only, after cluster is healthy)
+docker/seed-sites.sh
 ```

 ### After Code Changes
--- a/docker/deploy.sh
+++ b/docker/deploy.sh
@@ -26,4 +26,7 @@ echo "  Active node check:    http://localhost:9001/health/active"
 echo "  Traefik dashboard:    http://localhost:8180"
 echo "  Management API:       http://localhost:9000/management"
 echo ""
+echo "To seed test sites (first-time setup):"
+echo "  docker/seed-sites.sh"
+echo ""
 echo "Logs: docker compose -f $SCRIPT_DIR/docker-compose.yml logs -f"
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@@ -40,6 +40,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9021:8082"   # Akka remoting (host access for debugging)
+      - "9023:8083"   # gRPC streaming
    volumes:
      - ./site-a-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-a-node-a/data:/app/data
@@ -55,6 +56,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9022:8082"   # Akka remoting
+      - "9024:8083"   # gRPC streaming
    volumes:
      - ./site-a-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-a-node-b/data:/app/data
@@ -70,6 +72,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9031:8082"   # Akka remoting
+      - "9033:8083"   # gRPC streaming
    volumes:
      - ./site-b-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-b-node-a/data:/app/data
@@ -85,6 +88,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9032:8082"   # Akka remoting
+      - "9034:8083"   # gRPC streaming
    volumes:
      - ./site-b-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-b-node-b/data:/app/data
@@ -100,6 +104,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9041:8082"   # Akka remoting
+      - "9043:8083"   # gRPC streaming
    volumes:
      - ./site-c-node-a/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-c-node-a/data:/app/data
@@ -115,6 +120,7 @@ services:
      SCADALINK_CONFIG: Site
    ports:
      - "9042:8082"   # Akka remoting
+      - "9044:8083"   # gRPC streaming
    volumes:
      - ./site-c-node-b/appsettings.Site.json:/app/appsettings.Site.json:ro
      - ./site-c-node-b/data:/app/data
--- a/docker/seed-sites.sh
+++ b/docker/seed-sites.sh
@@ -0,0 +1,62 @@
+#!/bin/bash
+set -euo pipefail
+
+# Seed the three test sites with Akka and gRPC addresses.
+# Run after deploy.sh once the central cluster is healthy.
+#
+# Prerequisites:
+#   - Infrastructure services running (infra/docker-compose up -d)
+#   - Application containers running (docker/deploy.sh)
+#   - Central cluster healthy (curl http://localhost:9000/health/ready)
+#
+# Usage:
+#   docker/seed-sites.sh
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
+CLI="dotnet run --project $PROJECT_ROOT/src/ScadaLink.CLI --"
+AUTH="--username multi-role --password password"
+URL="--url http://localhost:9000"
+
+echo "=== Seeding ScadaLink Sites ==="
+
+echo ""
+echo "Creating Site-A (Test Plant A)..."
+$CLI $URL $AUTH site create \
+    --name "Test Plant A" \
+    --identifier "site-a" \
+    --description "Test site A - two-node cluster" \
+    --node-a-address "akka.tcp://scadalink@scadalink-site-a-a:8082" \
+    --node-b-address "akka.tcp://scadalink@scadalink-site-a-b:8082" \
+    --grpc-node-a-address "http://scadalink-site-a-a:8083" \
+    --grpc-node-b-address "http://scadalink-site-a-b:8083" \
+|| echo "  (Site-A may already exist)"
+
+echo ""
+echo "Creating Site-B (Test Plant B)..."
+$CLI $URL $AUTH site create \
+    --name "Test Plant B" \
+    --identifier "site-b" \
+    --description "Test site B - two-node cluster" \
+    --node-a-address "akka.tcp://scadalink@scadalink-site-b-a:8082" \
+    --node-b-address "akka.tcp://scadalink@scadalink-site-b-b:8082" \
+    --grpc-node-a-address "http://scadalink-site-b-a:8083" \
+    --grpc-node-b-address "http://scadalink-site-b-b:8083" \
+|| echo "  (Site-B may already exist)"
+
+echo ""
+echo "Creating Site-C (Test Plant C)..."
+$CLI $URL $AUTH site create \
+    --name "Test Plant C" \
+    --identifier "site-c" \
+    --description "Test site C - two-node cluster" \
+    --node-a-address "akka.tcp://scadalink@scadalink-site-c-a:8082" \
+    --node-b-address "akka.tcp://scadalink@scadalink-site-c-b:8082" \
+    --grpc-node-a-address "http://scadalink-site-c-a:8083" \
+    --grpc-node-b-address "http://scadalink-site-c-b:8083" \
+|| echo "  (Site-C may already exist)"
+
+echo ""
+echo "=== Site seeding complete ==="
+echo ""
+echo "Verify with: $CLI $URL $AUTH site list"
--- a/docker/site-a-node-a/appsettings.Site.json
+++ b/docker/site-a-node-a/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-a-a",
      "SiteId": "site-a",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [
--- a/docker/site-a-node-b/appsettings.Site.json
+++ b/docker/site-a-node-b/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-a-b",
      "SiteId": "site-a",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [
--- a/docker/site-b-node-a/appsettings.Site.json
+++ b/docker/site-b-node-a/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-b-a",
      "SiteId": "site-b",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [
--- a/docker/site-b-node-b/appsettings.Site.json
+++ b/docker/site-b-node-b/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-b-b",
      "SiteId": "site-b",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [
--- a/docker/site-c-node-a/appsettings.Site.json
+++ b/docker/site-c-node-a/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-c-a",
      "SiteId": "site-c",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [
--- a/docker/site-c-node-b/appsettings.Site.json
+++ b/docker/site-c-node-b/appsettings.Site.json
@@ -4,7 +4,8 @@
      "Role": "Site",
      "NodeHostname": "scadalink-site-c-b",
      "SiteId": "site-c",
-      "RemotingPort": 8082
+      "RemotingPort": 8082,
+      "GrpcPort": 8083
    },
    "Cluster": {
      "SeedNodes": [