dohertj2/scadalink-design
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: replace site registration with database-driven site addressing

Browse Source 
Central now resolves site Akka remoting addresses from the Sites DB table
(NodeAAddress/NodeBAddress) instead of relying on runtime RegisterSite
messages. Eliminates the race condition where sites starting before central
had their registration dead-lettered. Addresses are cached in
CentralCommunicationActor with 60s periodic refresh and on-demand refresh
when sites are added/edited/deleted via UI or CLI.
This commit is contained in:
Joseph Doherty
2026-03-17 23:13:10 -04:00
parent eb8d5ca2c0
commit 9e97c1acd2
21 changed files with 1641 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
CLAUDE.md
View File

@@ -11,8 +11,7 @@ This project contains design documentation for a distributed SCADA system built
- `AkkaDotNet/` — Akka.NET reference documentation and best practices notes.
- `test_infra.md` — Master test infrastructure doc (OPC UA, LDAP, MS SQL).
- `infra/` — Docker Compose and config files for local test services.
There is no source code in this project — only design documentation in markdown.
- `docker/` — Docker infrastructure for the 8-node cluster topology (2 central + 3 sites). See [`docker/README.md`](docker/README.md) for cluster setup, port allocation, and management commands.
## Document Conventions
@@ -79,6 +78,7 @@ There is no source code in this project — only design documentation in markdow
- Tag path resolution retried periodically for devices still booting.
- Static attribute writes persisted to local SQLite (survive restart/failover, reset on redeployment).
- All timestamps are UTC throughout the system.
- Site addressing is database-driven: NodeAAddress and NodeBAddress stored in the Sites table, cached in CentralCommunicationActor, refreshed periodically (60s) and on admin changes. Heartbeats serve health monitoring only.
### External Integrations
- External System Gateway: HTTP/REST only, JSON serialization, API key + Basic Auth.

13
Component-Communication.md
View File

@@ -10,6 +10,7 @@ Both central and site clusters. Each side has communication actors that handle m
## Responsibilities
- Resolve site addresses from the configuration database and maintain a cached address map.
- Establish and maintain Akka.NET remoting connections between central and each site cluster.
- Route messages between central and site clusters in a hub-and-spoke topology.
- Broker requests from external systems (via central) to sites and return responses.
@@ -82,6 +83,17 @@ Central Cluster
- Sites do **not** communicate with each other.
- All inter-cluster communication flows through central.
## Site Address Resolution
Central discovers site addresses through the **configuration database**, not runtime registration:
- Each site record in the Sites table includes optional **NodeAAddress** and **NodeBAddress** fields containing the Akka remoting paths of the site's cluster nodes.
- The **CentralCommunicationActor** loads all site addresses from the database at startup and caches them in memory.
- The cache is **refreshed every 60 seconds** and **on-demand** when site records are added, edited, or deleted via the Central UI or CLI.
- When routing a message to a site, the actor **prefers NodeA** and **falls back to NodeB** if NodeA is unreachable.
- **Heartbeats** from sites serve **health monitoring only** — they do not serve as a registration or address discovery mechanism.
- If no addresses are configured for a site, messages to that site are **dropped** and the caller's Ask times out.
## Message Timeouts
Each request/response pattern has a default timeout that can be overridden in configuration:
@@ -139,6 +151,7 @@ The ManagementActor is registered at the well-known path `/user/management` on c
- **Akka.NET Remoting**: Provides the transport layer.
- **Cluster Infrastructure**: Manages node roles and failover detection.
- **Configuration Database**: Provides site node addresses (NodeAAddress, NodeBAddress) for address resolution.
## Interactions

3
HighLevelReqs.md
View File

@@ -46,6 +46,7 @@
### 2.2 Communication: Central ↔ Site
- Central-to-site and site-to-central communication uses **Akka.NET** (remoting/cluster).
- **Site addressing**: Site Akka remoting addresses (NodeA and NodeB) are stored in the **Sites database table** and configured via the Central UI. Central resolves site addresses from the database (cached in memory, refreshed periodically and on admin changes) rather than relying on runtime registration messages from sites.
- **Central as integration hub**: Central brokers requests between external systems and sites. For example, a recipe manager sends a recipe to central, which routes it to the appropriate site. MES requests machine values from central, which routes the request to the site and returns the response.
- **Real-time data streaming** is not continuous for all machine data. The only real-time stream is an **on-demand debug view** — an engineer in the central UI can open a live view of a specific instance's tag values and alarm states for troubleshooting purposes. This is session-based and temporary. The debug view subscribes to the site-wide Akka stream filtered by instance (see Section 8.1).
@@ -360,7 +361,7 @@ The central cluster hosts a **configuration and management UI** (no live machine
- **Database Connection Management**: Define named database connections for script use.
- **Inbound API Management**: Manage API keys (create, enable/disable, delete). Define API methods (name, parameters, return values, approved keys, implementation script). *(Admin role for keys, Design role for methods.)*
- **Instance Management**: Create instances from templates, bind data connections (per-attribute, with **bulk assignment** UI for selecting multiple attributes and assigning a data connection at once), set instance-level attribute overrides, assign instances to areas. **Disable** or **delete** instances.
- **Site & Data Connection Management**: Define sites, manage data connections and assign them to sites.
- **Site & Data Connection Management**: Define sites (including optional NodeAAddress and NodeBAddress fields for Akka remoting paths), manage data connections and assign them to sites.
- **Area Management**: Define hierarchical area structures per site for organizing instances.
- **Deployment**: View diffs between deployed and current template-derived configurations, deploy updates to individual instances. Filter instances by area. Pre-deployment validation runs automatically before any deployment is sent.
- **System-Wide Artifact Deployment**: Explicitly deploy shared scripts, external system definitions, database connection definitions, data connection definitions, notification lists, and SMTP configuration to all sites or to an individual site (requires Deployment role). Per-site deployment is available via the Sites admin page.

2
docker/Dockerfile
View File

@@ -30,7 +30,7 @@ RUN dotnet restore src/ScadaLink.Host/ScadaLink.Host.csproj
FROM restore AS build
COPY src/ src/
RUN dotnet publish src/ScadaLink.Host/ScadaLink.Host.csproj \
-c Release -o /app/publish --no-restore
-c Release -o /app/publish
# Stage 3: Runtime (minimal image, no SDK)
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS runtime

2
docker/docker-compose.yml
View File

@@ -4,6 +4,7 @@ services:
container_name: scadalink-central-a
environment:
SCADALINK_CONFIG: Central
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: "http://+:5000"
ports:
- "9001:5000" # Web UI + Inbound API
@@ -20,6 +21,7 @@ services:
container_name: scadalink-central-b
environment:
SCADALINK_CONFIG: Central
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: "http://+:5000"
ports:
- "9002:5000" # Web UI + Inbound API

37
src/ScadaLink.CLI/Commands/SiteCommands.cs
View File

@@ -12,6 +12,7 @@ public static class SiteCommands
command.Add(BuildList(contactPointsOption, formatOption));
command.Add(BuildCreate(contactPointsOption, formatOption));
command.Add(BuildUpdate(contactPointsOption, formatOption));
command.Add(BuildDelete(contactPointsOption, formatOption));
command.Add(BuildDeployArtifacts(contactPointsOption, formatOption));
@@ -34,19 +35,53 @@ public static class SiteCommands
var nameOption = new Option<string>("--name") { Description = "Site name", Required = true };
var identifierOption = new Option<string>("--identifier") { Description = "Site identifier", Required = true };
var descOption = new Option<string?>("--description") { Description = "Site description" };
var nodeAOption = new Option<string?>("--node-a-address") { Description = "Akka address for Node A" };
var nodeBOption = new Option<string?>("--node-b-address") { Description = "Akka address for Node B" };
var cmd = new Command("create") { Description = "Create a new site" };
cmd.Add(nameOption);
cmd.Add(identifierOption);
cmd.Add(descOption);
cmd.Add(nodeAOption);
cmd.Add(nodeBOption);
cmd.SetAction(async (ParseResult result) =>
{
var name = result.GetValue(nameOption)!;
var identifier = result.GetValue(identifierOption)!;
var desc = result.GetValue(descOption);
var nodeA = result.GetValue(nodeAOption);
var nodeB = result.GetValue(nodeBOption);
return await CommandHelpers.ExecuteCommandAsync(
result, contactPointsOption, formatOption,
new CreateSiteCommand(name, identifier, desc));
new CreateSiteCommand(name, identifier, desc, nodeA, nodeB));
});
return cmd;
}
private static Command BuildUpdate(Option<string> contactPointsOption, Option<string> formatOption)
{
var idOption = new Option<int>("--id") { Description = "Site ID", Required = true };
var nameOption = new Option<string>("--name") { Description = "Site name", Required = true };
var descOption = new Option<string?>("--description") { Description = "Site description" };
var nodeAOption = new Option<string?>("--node-a-address") { Description = "Akka address for Node A" };
var nodeBOption = new Option<string?>("--node-b-address") { Description = "Akka address for Node B" };
var cmd = new Command("update") { Description = "Update an existing site" };
cmd.Add(idOption);
cmd.Add(nameOption);
cmd.Add(descOption);
cmd.Add(nodeAOption);
cmd.Add(nodeBOption);
cmd.SetAction(async (ParseResult result) =>
{
var id = result.GetValue(idOption);
var name = result.GetValue(nameOption)!;
var desc = result.GetValue(descOption);
var nodeA = result.GetValue(nodeAOption);
var nodeB = result.GetValue(nodeBOption);
return await CommandHelpers.ExecuteCommandAsync(
result, contactPointsOption, formatOption,
new UpdateSiteCommand(id, name, desc, nodeA, nodeB));
});
return cmd;
}

34
src/ScadaLink.CentralUI/Components/Pages/Admin/Sites.razor
View File

@@ -2,10 +2,12 @@
@using ScadaLink.Security
@using ScadaLink.Commons.Entities.Sites
@using ScadaLink.Commons.Interfaces.Repositories
@using ScadaLink.Communication
@using ScadaLink.DeploymentManager
@attribute [Authorize(Policy = AuthorizationPolicies.RequireAdmin)]
@inject ISiteRepository SiteRepository
@inject ArtifactDeploymentService ArtifactDeploymentService
@inject CommunicationService CommunicationService
@inject AuthenticationStateProvider AuthStateProvider
<div class="container-fluid mt-3">
@@ -61,6 +63,18 @@
<button class="btn btn-outline-secondary btn-sm" @onclick="CancelForm">Cancel</button>
</div>
</div>
<div class="row g-2 align-items-end mt-1">
<div class="col-md-6">
<label class="form-label small">Node A Address</label>
<input type="text" class="form-control form-control-sm" @bind="_formNodeAAddress"
placeholder="akka.tcp://scadalink@host:port/user/site-communication" />
</div>
<div class="col-md-6">
<label class="form-label small">Node B Address (optional)</label>
<input type="text" class="form-control form-control-sm" @bind="_formNodeBAddress"
placeholder="akka.tcp://scadalink@host:port/user/site-communication" />
</div>
</div>
@if (_formError != null)
{
<div class="text-danger small mt-1">@_formError</div>
@@ -76,6 +90,8 @@
<th>Name</th>
<th>Identifier</th>
<th>Description</th>
<th>Node A</th>
<th>Node B</th>
<th>Data Connections</th>
<th style="width: 260px;">Actions</th>
</tr>
@@ -84,7 +100,7 @@
@if (_sites.Count == 0)
{
<tr>
<td colspan="6" class="text-muted text-center">No sites configured.</td>
<td colspan="8" class="text-muted text-center">No sites configured.</td>
</tr>
}
@foreach (var site in _sites)
@@ -94,6 +110,8 @@
<td>@site.Name</td>
<td><code>@site.SiteIdentifier</code></td>
<td class="text-muted small">@(site.Description ?? "—")</td>
<td class="small text-truncate" style="max-width: 200px;" title="@site.NodeAAddress">@(site.NodeAAddress ?? "—")</td>
<td class="small text-truncate" style="max-width: 200px;" title="@site.NodeBAddress">@(site.NodeBAddress ?? "—")</td>
<td>
@{
var conns = _siteConnections.GetValueOrDefault(site.Id);
@@ -143,6 +161,8 @@
private string _formName = string.Empty;
private string _formIdentifier = string.Empty;
private string? _formDescription;
private string? _formNodeAAddress;
private string? _formNodeBAddress;
private string? _formError;
private bool _deploying;
@@ -185,6 +205,8 @@
_formName = string.Empty;
_formIdentifier = string.Empty;
_formDescription = null;
_formNodeAAddress = null;
_formNodeBAddress = null;
_formError = null;
_showForm = true;
}
@@ -195,6 +217,8 @@
_formName = site.Name;
_formIdentifier = site.SiteIdentifier;
_formDescription = site.Description;
_formNodeAAddress = site.NodeAAddress;
_formNodeBAddress = site.NodeBAddress;
_formError = null;
_showForm = true;
}
@@ -222,6 +246,8 @@
{
_editingSite.Name = _formName.Trim();
_editingSite.Description = _formDescription?.Trim();
_editingSite.NodeAAddress = _formNodeAAddress?.Trim();
_editingSite.NodeBAddress = _formNodeBAddress?.Trim();
await SiteRepository.UpdateSiteAsync(_editingSite);
}
else
@@ -233,12 +259,15 @@
}
var site = new Site(_formName.Trim(), _formIdentifier.Trim())
{
Description = _formDescription?.Trim()
Description = _formDescription?.Trim(),
NodeAAddress = _formNodeAAddress?.Trim(),
NodeBAddress = _formNodeBAddress?.Trim()
};
await SiteRepository.AddSiteAsync(site);
}
await SiteRepository.SaveChangesAsync();
CommunicationService.RefreshSiteAddresses();
_showForm = false;
_editingSite = null;
_toast.ShowSuccess(_editingSite == null ? "Site created." : "Site updated.");
@@ -262,6 +291,7 @@
{
await SiteRepository.DeleteSiteAsync(site.Id);
await SiteRepository.SaveChangesAsync();
CommunicationService.RefreshSiteAddresses();
_toast.ShowSuccess($"Site '{site.Name}' deleted.");
await LoadDataAsync();
}

2
src/ScadaLink.Commons/Entities/Sites/Site.cs
View File

@@ -6,6 +6,8 @@ public class Site
public string Name { get; set; }
public string SiteIdentifier { get; set; }
public string? Description { get; set; }
public string? NodeAAddress { get; set; }
public string? NodeBAddress { get; set; }
public Site(string name, string siteIdentifier)
{

4
src/ScadaLink.Commons/Messages/Management/SiteCommands.cs
View File

@@ -2,8 +2,8 @@ namespace ScadaLink.Commons.Messages.Management;
public record ListSitesCommand;
public record GetSiteCommand(int SiteId);
public record CreateSiteCommand(string Name, string SiteIdentifier, string? Description);
public record UpdateSiteCommand(int SiteId, string Name, string? Description);
public record CreateSiteCommand(string Name, string SiteIdentifier, string? Description, string? NodeAAddress = null, string? NodeBAddress = null);
public record UpdateSiteCommand(int SiteId, string Name, string? Description, string? NodeAAddress = null, string? NodeBAddress = null);
public record DeleteSiteCommand(int SiteId);
public record ListAreasCommand(int SiteId);
public record CreateAreaCommand(int SiteId, string Name, int? ParentAreaId);

117
src/ScadaLink.Communication/Actors/CentralCommunicationActor.cs
View File

@@ -1,5 +1,7 @@
using Akka.Actor;
using Akka.Event;
using Microsoft.Extensions.DependencyInjection;
using ScadaLink.Commons.Interfaces.Repositories;
using ScadaLink.Commons.Messages.Communication;
using ScadaLink.Commons.Messages.Health;
@@ -7,7 +9,7 @@ namespace ScadaLink.Communication.Actors;
/// <summary>
/// Central-side actor that routes messages from central to site clusters via Akka remoting.
/// Maintains a registry of known site actor paths (learned from heartbeats/connection events).
/// Resolves site addresses from the database on a periodic refresh cycle.
///
/// WP-4: All 8 message patterns routed through this actor.
/// WP-5: Ask timeout on connection drop (no central buffering). Debug streams killed on interruption.
@@ -15,12 +17,14 @@ namespace ScadaLink.Communication.Actors;
public class CentralCommunicationActor : ReceiveActor
{
private readonly ILoggingAdapter _log = Context.GetLogger();
private readonly IServiceProvider _serviceProvider;
/// <summary>
/// Maps SiteId → remote SiteCommunicationActor selection.
/// Updated when heartbeats arrive or connection state changes.
/// Cached site address entries loaded from the database.
/// Maps SiteIdentifier → (NodeA selection, NodeB selection).
/// Refreshed periodically via RefreshSiteAddresses.
/// </summary>
private readonly Dictionary<string, ActorSelection> _siteSelections = new();
private Dictionary<string, (ActorSelection? NodeA, ActorSelection? NodeB)> _siteAddressCache = new();
/// <summary>
/// Tracks active debug view subscriptions: correlationId → (siteId, subscriber).
@@ -34,31 +38,30 @@ public class CentralCommunicationActor : ReceiveActor
/// </summary>
private readonly Dictionary<string, string> _inProgressDeployments = new();
public CentralCommunicationActor()
private ICancelable? _refreshSchedule;
public CentralCommunicationActor(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
// Site address cache loaded from database
Receive<SiteAddressCacheLoaded>(HandleSiteAddressCacheLoaded);
// Periodic refresh trigger
Receive<RefreshSiteAddresses>(_ => LoadSiteAddressesFromDb());
// Site registration via heartbeats
Receive<HeartbeatMessage>(HandleHeartbeat);
// Connection state changes
Receive<ConnectionStateChanged>(HandleConnectionStateChanged);
// Site registration command (manual or from discovery)
Receive<RegisterSite>(HandleRegisterSite);
// Route enveloped messages to sites
Receive<SiteEnvelope>(HandleSiteEnvelope);
}
private void HandleHeartbeat(HeartbeatMessage heartbeat)
{
// Heartbeats arrive from sites — forward to any interested central actors
// The sender's path tells us the site's communication actor address
if (!_siteSelections.ContainsKey(heartbeat.SiteId))
{
var senderPath = Sender.Path.ToString();
_log.Info("Learned site {0} from heartbeat at path {1}", heartbeat.SiteId, senderPath);
}
// Forward heartbeat to parent/subscribers (central health monitoring)
Context.Parent.Tell(heartbeat);
}
@@ -94,7 +97,7 @@ public class CentralCommunicationActor : ReceiveActor
_inProgressDeployments.Remove(deploymentId);
}
_siteSelections.Remove(msg.SiteId);
// Note: Do NOT remove from _siteAddressCache — addresses are persistent in the database
}
else
{
@@ -102,30 +105,71 @@ public class CentralCommunicationActor : ReceiveActor
}
}
private void HandleRegisterSite(RegisterSite msg)
{
var selection = Context.ActorSelection(msg.RemoteActorPath);
_siteSelections[msg.SiteId] = selection;
_log.Info("Registered site {0} at path {1}", msg.SiteId, msg.RemoteActorPath);
}
private void HandleSiteEnvelope(SiteEnvelope envelope)
{
if (!_siteSelections.TryGetValue(envelope.SiteId, out var siteSelection))
if (!_siteAddressCache.TryGetValue(envelope.SiteId, out var entry))
{
_log.Warning("No known path for site {0}, cannot route message {1}",
_log.Warning("No known address for site {0}, cannot route message {1}",
envelope.SiteId, envelope.Message.GetType().Name);
// The Ask will timeout on the caller side — no central buffering (WP-5)
return;
}
// Prefer NodeA, fall back to NodeB
var selection = entry.NodeA ?? entry.NodeB;
if (selection == null)
{
_log.Warning("Site {0} has no configured node addresses, cannot route message {1}",
envelope.SiteId, envelope.Message.GetType().Name);
return;
}
// Track debug subscriptions for cleanup on disconnect
TrackMessageForCleanup(envelope);
// Forward the inner message to the site, preserving the original sender
// so the site can reply directly to the caller (completing the Ask pattern)
siteSelection.Tell(envelope.Message, Sender);
selection.Tell(envelope.Message, Sender);
}
private void LoadSiteAddressesFromDb()
{
var self = Self;
Task.Run(async () =>
{
using var scope = _serviceProvider.CreateScope();
var repo = scope.ServiceProvider.GetRequiredService<ISiteRepository>();
var sites = await repo.GetAllSitesAsync();
var cache = new Dictionary<string, (string? NodeAAddress, string? NodeBAddress)>();
foreach (var site in sites)
{
if (string.IsNullOrWhiteSpace(site.NodeAAddress) && string.IsNullOrWhiteSpace(site.NodeBAddress))
continue;
cache[site.SiteIdentifier] = (site.NodeAAddress, site.NodeBAddress);
}
return new SiteAddressCacheLoaded(cache);
}).PipeTo(self);
}
private void HandleSiteAddressCacheLoaded(SiteAddressCacheLoaded msg)
{
var newCache = new Dictionary<string, (ActorSelection? NodeA, ActorSelection? NodeB)>();
foreach (var (siteId, (nodeAAddr, nodeBAddr)) in msg.Addresses)
{
var nodeA = !string.IsNullOrWhiteSpace(nodeAAddr)
? Context.ActorSelection(nodeAAddr)
: null;
var nodeB = !string.IsNullOrWhiteSpace(nodeBAddr)
? Context.ActorSelection(nodeBAddr)
: null;
newCache[siteId] = (nodeA, nodeB);
}
_siteAddressCache = newCache;
_log.Info("Site address cache refreshed with {0} site(s)", _siteAddressCache.Count);
}
private void TrackMessageForCleanup(SiteEnvelope envelope)
@@ -149,11 +193,20 @@ public class CentralCommunicationActor : ReceiveActor
protected override void PreStart()
{
_log.Info("CentralCommunicationActor started");
// Schedule periodic refresh of site addresses from the database
_refreshSchedule = Context.System.Scheduler.ScheduleTellRepeatedlyCancelable(
TimeSpan.Zero,
TimeSpan.FromSeconds(60),
Self,
new RefreshSiteAddresses(),
ActorRefs.NoSender);
}
protected override void PostStop()
{
_log.Info("CentralCommunicationActor stopped. In-progress deployments treated as failed (WP-5).");
_refreshSchedule?.Cancel();
// On central failover, all in-progress deployments are failed
_inProgressDeployments.Clear();
_debugSubscriptions.Clear();
@@ -161,9 +214,15 @@ public class CentralCommunicationActor : ReceiveActor
}
/// <summary>
/// Command to register a site's remote communication actor path.
/// Command to trigger a refresh of site addresses from the database.
/// </summary>
public record RegisterSite(string SiteId, string RemoteActorPath);
public record RefreshSiteAddresses;
/// <summary>
/// Internal message carrying the loaded site address data from the database.
/// ActorSelection creation happens on the actor thread in HandleSiteAddressCacheLoaded.
/// </summary>
internal record SiteAddressCacheLoaded(Dictionary<string, (string? NodeAAddress, string? NodeBAddress)> Addresses);
/// <summary>
/// Notification sent to debug view subscribers when the stream is terminated

9
src/ScadaLink.Communication/CommunicationService.cs
View File

@@ -9,6 +9,7 @@ using ScadaLink.Commons.Messages.InboundApi;
using ScadaLink.Commons.Messages.Integration;
using ScadaLink.Commons.Messages.Lifecycle;
using ScadaLink.Commons.Messages.RemoteQuery;
using ScadaLink.Communication.Actors;
namespace ScadaLink.Communication;
@@ -39,6 +40,14 @@ public class CommunicationService
_centralCommunicationActor = centralCommunicationActor;
}
/// <summary>
/// Triggers an immediate refresh of the site address cache from the database.
/// </summary>
public void RefreshSiteAddresses()
{
GetActor().Tell(new RefreshSiteAddresses());
}
private IActorRef GetActor()
{
return _centralCommunicationActor

3
src/ScadaLink.ConfigurationDatabase/Configurations/SiteConfiguration.cs
View File

@@ -21,6 +21,9 @@ public class SiteConfiguration : IEntityTypeConfiguration<Site>
builder.Property(s => s.Description)
.HasMaxLength(2000);
builder.Property(s => s.NodeAAddress).HasMaxLength(500);
builder.Property(s => s.NodeBAddress).HasMaxLength(500);
builder.HasIndex(s => s.Name).IsUnique();
builder.HasIndex(s => s.SiteIdentifier).IsUnique();
}

1248
src/ScadaLink.ConfigurationDatabase/Migrations/20260318025613_AddSiteNodeAddresses.Designer.cs generated Normal file
View File

File diff suppressed because it is too large Load Diff

42
src/ScadaLink.ConfigurationDatabase/Migrations/20260318025613_AddSiteNodeAddresses.cs Normal file
View File

@@ -0,0 +1,42 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
#pragma warning disable CA1814 // Prefer jagged arrays over multidimensional
namespace ScadaLink.ConfigurationDatabase.Migrations
{
/// <inheritdoc />
public partial class AddSiteNodeAddresses : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "NodeAAddress",
table: "Sites",
type: "nvarchar(500)",
maxLength: 500,
nullable: true);
migrationBuilder.AddColumn<string>(
name: "NodeBAddress",
table: "Sites",
type: "nvarchar(500)",
maxLength: 500,
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "NodeAAddress",
table: "Sites");
migrationBuilder.DropColumn(
name: "NodeBAddress",
table: "Sites");
}
}
}

26
src/ScadaLink.ConfigurationDatabase/Migrations/ScadaLinkDbContextModelSnapshot.cs
View File

@@ -699,6 +699,24 @@ namespace ScadaLink.ConfigurationDatabase.Migrations
Id = 1,
LdapGroupName = "SCADA-Admins",
Role = "Admin"
},
new
{
Id = 2,
LdapGroupName = "SCADA-Designers",
Role = "Design"
},
new
{
Id = 3,
LdapGroupName = "SCADA-Deploy-All",
Role = "Deployment"
},
new
{
Id = 4,
LdapGroupName = "SCADA-Deploy-SiteA",
Role = "Deployment"
});
});
@@ -773,6 +791,14 @@ namespace ScadaLink.ConfigurationDatabase.Migrations
.HasMaxLength(200)
.HasColumnType("nvarchar(200)");
b.Property<string>("NodeAAddress")
.HasMaxLength(500)
.HasColumnType("nvarchar(500)");
b.Property<string>("NodeBAddress")
.HasMaxLength(500)
.HasColumnType("nvarchar(500)");
b.Property<string>("SiteIdentifier")
.IsRequired()
.HasMaxLength(100)

9
src/ScadaLink.Host/Actors/AkkaHostedService.cs
View File

@@ -169,7 +169,7 @@ akka {{
private void RegisterCentralActors()
{
var centralCommActor = _actorSystem!.ActorOf(
Props.Create(() => new CentralCommunicationActor()),
Props.Create(() => new CentralCommunicationActor(_serviceProvider)),
"central-communication");
// Wire up the CommunicationService with the actor reference
@@ -264,13 +264,8 @@ akka {{
var siteCommActor = _actorSystem.ActorSelection("/user/site-communication");
siteCommActor.Tell(new RegisterCentralPath(_communicationOptions.CentralActorPath));
// Also register this site with Central so it knows our address
var centralSelection = _actorSystem.ActorSelection(_communicationOptions.CentralActorPath);
var localSiteCommPath = $"akka.tcp://scadalink@{_nodeOptions.NodeHostname}:{_nodeOptions.RemotingPort}/user/site-communication";
centralSelection.Tell(new RegisterSite(_nodeOptions.SiteId!, localSiteCommPath));
_logger.LogInformation(
"Registered with Central at {CentralPath} as site {SiteId}",
"Configured central heartbeat path at {CentralPath} for site {SiteId}",
_communicationOptions.CentralActorPath, _nodeOptions.SiteId);
}
}

13
src/ScadaLink.Host/Components/App.razor
View File

@@ -83,11 +83,14 @@
<script src="/_framework/blazor.web.js"></script>
<script>
// Reconnection overlay for failover behavior
if (Blazor) {
Blazor.addEventListener('enhancedload', () => {
document.getElementById('reconnect-modal').style.display = 'none';
});
}
// Blazor object is available after blazor.web.js initializes
document.addEventListener('DOMContentLoaded', () => {
if (typeof Blazor !== 'undefined') {
Blazor.addEventListener('enhancedload', () => {
document.getElementById('reconnect-modal').style.display = 'none';
});
}
});
</script>
<script src="/lib/bootstrap/js/bootstrap.bundle.min.js"></script>
</body>

1
src/ScadaLink.Host/Program.cs
View File

@@ -130,6 +130,7 @@ try
ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});
app.MapStaticAssets();
app.MapCentralUI<ScadaLink.Host.Components.App>();
app.MapInboundAPI();
await app.RunAsync();

16
src/ScadaLink.ManagementService/ManagementActor.cs
View File

@@ -13,6 +13,7 @@ using ScadaLink.Commons.Interfaces.Repositories;
using ScadaLink.Commons.Messages.Management;
using ScadaLink.DeploymentManager;
using ScadaLink.HealthMonitoring;
using ScadaLink.Communication;
using ScadaLink.TemplateEngine;
using ScadaLink.TemplateEngine.Services;
@@ -320,9 +321,16 @@ public class ManagementActor : ReceiveActor
private static async Task<object?> HandleCreateSite(IServiceProvider sp, CreateSiteCommand cmd)
{
var repo = sp.GetRequiredService<ISiteRepository>();
var site = new Site(cmd.Name, cmd.SiteIdentifier) { Description = cmd.Description };
var site = new Site(cmd.Name, cmd.SiteIdentifier)
{
Description = cmd.Description,
NodeAAddress = cmd.NodeAAddress,
NodeBAddress = cmd.NodeBAddress
};
await repo.AddSiteAsync(site);
await repo.SaveChangesAsync();
var commService = sp.GetService<CommunicationService>();
commService?.RefreshSiteAddresses();
return site;
}
@@ -333,8 +341,12 @@ public class ManagementActor : ReceiveActor
?? throw new InvalidOperationException($"Site with ID {cmd.SiteId} not found.");
site.Name = cmd.Name;
site.Description = cmd.Description;
site.NodeAAddress = cmd.NodeAAddress;
site.NodeBAddress = cmd.NodeBAddress;
await repo.UpdateSiteAsync(site);
await repo.SaveChangesAsync();
var commService = sp.GetService<CommunicationService>();
commService?.RefreshSiteAddresses();
return site;
}
@@ -348,6 +360,8 @@ public class ManagementActor : ReceiveActor
$"Cannot delete site {cmd.SiteId}: it has {instances.Count} instance(s).");
await repo.DeleteSiteAsync(cmd.SiteId);
await repo.SaveChangesAsync();
var commService = sp.GetService<CommunicationService>();
commService?.RefreshSiteAddresses();
return true;
}

147
tests/ScadaLink.Communication.Tests/CentralCommunicationActorTests.cs
View File

@@ -1,5 +1,9 @@
using Akka.Actor;
using Akka.TestKit.Xunit2;
using Microsoft.Extensions.DependencyInjection;
using NSubstitute;
using ScadaLink.Commons.Entities.Sites;
using ScadaLink.Commons.Interfaces.Repositories;
using ScadaLink.Commons.Messages.Communication;
using ScadaLink.Commons.Messages.Deployment;
using ScadaLink.Commons.Messages.DebugView;
@@ -9,97 +13,158 @@ using ScadaLink.Communication.Actors;
namespace ScadaLink.Communication.Tests;
/// <summary>
/// WP-4: Tests for CentralCommunicationActor message routing.
/// WP-5: Tests for connection failure and failover handling.
/// Tests for CentralCommunicationActor with database-driven site addressing.
/// WP-4: Message routing via site address cache loaded from DB.
/// WP-5: Connection failure and failover handling.
/// </summary>
public class CentralCommunicationActorTests : TestKit
{
public CentralCommunicationActorTests()
: base(@"akka.loglevel = DEBUG")
public CentralCommunicationActorTests() : base(@"akka.loglevel = DEBUG") { }
private (IActorRef actor, ISiteRepository mockRepo) CreateActorWithMockRepo(IEnumerable<Site>? sites = null)
{
var mockRepo = Substitute.For<ISiteRepository>();
mockRepo.GetAllSitesAsync(Arg.Any<CancellationToken>())
.Returns(sites?.ToList() ?? new List<Site>());
var services = new ServiceCollection();
services.AddScoped(_ => mockRepo);
var sp = services.BuildServiceProvider();
var actor = Sys.ActorOf(Props.Create(() => new CentralCommunicationActor(sp)));
return (actor, mockRepo);
}
private Site CreateSite(string identifier, string? nodeAPath, string? nodeBPath = null) =>
new("Test Site", identifier) { NodeAAddress = nodeAPath, NodeBAddress = nodeBPath };
[Fact]
public void RegisterSite_AllowsMessageRouting()
public void DatabaseDrivenRouting_RoutesToConfiguredSite()
{
var centralActor = Sys.ActorOf(Props.Create(() => new CentralCommunicationActor()));
// Register a site pointing to the test probe
var probe = CreateTestProbe();
centralActor.Tell(new RegisterSite("site1", probe.Ref.Path.ToString()));
var site = CreateSite("site1", probe.Ref.Path.ToString());
var (actor, _) = CreateActorWithMockRepo(new[] { site });
// Send explicit refresh and wait for async DB load + PipeTo
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
// Send a message to the site
var command = new DeployInstanceCommand(
"dep1", "inst1", "hash1", "{}", "admin", DateTimeOffset.UtcNow);
centralActor.Tell(new SiteEnvelope("site1", command));
actor.Tell(new SiteEnvelope("site1", command));
// The probe should receive the inner message (not the envelope)
probe.ExpectMsg<DeployInstanceCommand>(msg => msg.DeploymentId == "dep1");
}
[Fact]
public void UnregisteredSite_MessageIsDropped()
public void UnconfiguredSite_MessageIsDropped()
{
var centralActor = Sys.ActorOf(Props.Create(() => new CentralCommunicationActor()));
var (actor, _) = CreateActorWithMockRepo();
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
var command = new DeployInstanceCommand(
"dep1", "inst1", "hash1", "{}", "admin", DateTimeOffset.UtcNow);
centralActor.Tell(new SiteEnvelope("unknown-site", command));
actor.Tell(new SiteEnvelope("unknown-site", command));
// No crash, no response — the ask will timeout on the caller side
ExpectNoMsg(TimeSpan.FromMilliseconds(200));
}
[Fact]
public void ConnectionLost_DebugStreamsKilled()
{
var centralActor = Sys.ActorOf(Props.Create(() => new CentralCommunicationActor()));
var siteProbe = CreateTestProbe();
var site = CreateSite("site1", siteProbe.Ref.Path.ToString());
var (actor, _) = CreateActorWithMockRepo(new[] { site });
// Register site
centralActor.Tell(new RegisterSite("site1", siteProbe.Ref.Path.ToString()));
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
// Subscribe to debug view (this tracks the subscription)
// Subscribe to debug view (tracks the subscription)
var subscriberProbe = CreateTestProbe();
var subRequest = new SubscribeDebugViewRequest("inst1", "corr-123");
centralActor.Tell(new SiteEnvelope("site1", subRequest), subscriberProbe.Ref);
actor.Tell(new SiteEnvelope("site1", subRequest), subscriberProbe.Ref);
// Simulate site disconnection
centralActor.Tell(new ConnectionStateChanged("site1", false, DateTimeOffset.UtcNow));
actor.Tell(new ConnectionStateChanged("site1", false, DateTimeOffset.UtcNow));
// The subscriber should receive a DebugStreamTerminated notification
subscriberProbe.ExpectMsg<DebugStreamTerminated>(
msg => msg.SiteId == "site1" && msg.CorrelationId == "corr-123");
}
[Fact]
public void ConnectionLost_SiteSelectionRemoved()
{
var centralActor = Sys.ActorOf(Props.Create(() => new CentralCommunicationActor()));
var siteProbe = CreateTestProbe();
centralActor.Tell(new RegisterSite("site1", siteProbe.Ref.Path.ToString()));
// Disconnect
centralActor.Tell(new ConnectionStateChanged("site1", false, DateTimeOffset.UtcNow));
// Sending a message to the disconnected site should be dropped
centralActor.Tell(new SiteEnvelope("site1",
new DeployInstanceCommand("dep2", "inst2", "hash2", "{}", "admin", DateTimeOffset.UtcNow)));
siteProbe.ExpectNoMsg(TimeSpan.FromMilliseconds(200));
}
[Fact]
public void Heartbeat_ForwardedToParent()
{
// Actor still needs IServiceProvider even though this test doesn't use routing
var mockRepo = Substitute.For<ISiteRepository>();
mockRepo.GetAllSitesAsync(Arg.Any<CancellationToken>())
.Returns(new List<Site>());
var services = new ServiceCollection();
services.AddScoped(_ => mockRepo);
var sp = services.BuildServiceProvider();
var parentProbe = CreateTestProbe();
var centralActor = parentProbe.ChildActorOf(
Props.Create(() => new CentralCommunicationActor()));
Props.Create(() => new CentralCommunicationActor(sp)));
var heartbeat = new HeartbeatMessage("site1", "host1", true, DateTimeOffset.UtcNow);
centralActor.Tell(heartbeat);
parentProbe.ExpectMsg<HeartbeatMessage>(msg => msg.SiteId == "site1");
}
[Fact]
public void RefreshSiteAddresses_UpdatesCache()
{
var probe1 = CreateTestProbe();
var probe2 = CreateTestProbe();
var site1 = CreateSite("site1", probe1.Ref.Path.ToString());
var (actor, mockRepo) = CreateActorWithMockRepo(new[] { site1 });
// Wait for initial load, then send explicit refresh
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
// Verify routing to site1 works
var cmd1 = new DeployInstanceCommand(
"dep1", "inst1", "hash1", "{}", "admin", DateTimeOffset.UtcNow);
actor.Tell(new SiteEnvelope("site1", cmd1));
probe1.ExpectMsg<DeployInstanceCommand>(msg => msg.DeploymentId == "dep1");
// Update mock repo to return both sites
var site2 = CreateSite("site2", probe2.Ref.Path.ToString());
mockRepo.GetAllSitesAsync(Arg.Any<CancellationToken>())
.Returns(new List<Site> { site1, site2 });
// Refresh again
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
// Verify routing to site2 now works
var cmd2 = new DeployInstanceCommand(
"dep2", "inst2", "hash2", "{}", "admin", DateTimeOffset.UtcNow);
actor.Tell(new SiteEnvelope("site2", cmd2));
probe2.ExpectMsg<DeployInstanceCommand>(msg => msg.DeploymentId == "dep2");
}
[Fact]
public void NodeBFallback_WhenNodeANotConfigured()
{
var probe = CreateTestProbe();
var site = CreateSite("site1", null, probe.Ref.Path.ToString());
var (actor, _) = CreateActorWithMockRepo(new[] { site });
actor.Tell(new RefreshSiteAddresses());
Thread.Sleep(1000);
var command = new DeployInstanceCommand(
"dep1", "inst1", "hash1", "{}", "admin", DateTimeOffset.UtcNow);
actor.Tell(new SiteEnvelope("site1", command));
probe.ExpectMsg<DeployInstanceCommand>(msg => msg.DeploymentId == "dep1");
}
}

1
tests/ScadaLink.Communication.Tests/ScadaLink.Communication.Tests.csproj
View File

@@ -12,6 +12,7 @@
<PackageReference Include="Akka.TestKit.Xunit2" Version="1.5.62" />
<PackageReference Include="coverlet.collector" Version="6.0.4" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.5" />
<PackageReference Include="NSubstitute" Version="5.3.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.4" />