feat(central): one central cache-invalidation topic; role-mapping and external-definition caches follow peer edits (ScadaBridge#100 item 3, ScadaBridge#83)

The #96 site-psk-invalidation topic was the first cross-node invalidation broadcast.
It is generalised into ONE central DistributedPubSub topic, central-cache-invalidation,
carrying typed messages (Commons ICentralCacheInvalidation: SitePskInvalidated,
RoleMappingsInvalidated, ExternalDefinitionsInvalidated, InboundApiMethodInvalidated)
in an envelope stamped with the publisher's process-unique origin id.

- Host: CentralCacheInvalidationBus (ICentralCacheInvalidationPublisher, no-op until
  the mediator is set, never throws) and CentralCacheInvalidationSubscriber, which
  skips its own node's envelopes and hands the peer's to every registered
  ICentralCacheInvalidationHandler (a failing handler does not stop the rest).
  Replaces SitePskInvalidationSubscriber.
- Migration of #96: SitePskRotationInvalidator publishes through the bus and is the
  PSK handler; SitePskInvalidated keeps its type name and is also published bare on
  site-psk-invalidation, which the subscriber still listens to, so a mixed-version
  central pair keeps exchanging PSK invalidations. Remove after one release.
- Audit D1: RoleMappingCache is a handler; ManagementActor's post-commit role-mapping
  invalidation also publishes RoleMappingsInvalidated, so a revocation is immediate on
  the peer instead of after RoleMappingCacheSeconds.
- Audit D3 / #83 (peer half): ExternalDefinitionLookupCache is a handler; the 9
  ManagementActor handlers and the Transport bundle importer publish
  ExternalDefinitionsInvalidated after commit. (#83's local import half was already in.)

TTLs remain the fallback for a lost broadcast or a direct SQL edit. Tests: bus and
subscriber unit tests (origin filter, legacy topic), peer-node tests through the real
caches in ManagementService.Tests, importer publish, composition root, and a real
two-node cluster from the production HOCON (CentralCacheInvalidationTwoNodeTests)
proving every message type crosses with the production serializer. Docs: Host,
Communication, Security, ManagementService, ESG, and the #85 cache audit.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 15:50:03 -05:00
parent 063fb21815
commit 153609ef8a
27 changed files with 794 additions and 110 deletions
@@ -17,6 +17,7 @@ using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Scripting;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Communication;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Transport;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Scripting;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Transport;
@@ -98,6 +99,9 @@ public sealed class BundleImporter : IBundleImporter
// Optional (Gitea #83): drops this node's external-definition lookup cache after a
// committed import. Null on hosts without the External System Gateway.
private readonly IExternalDefinitionCacheInvalidator? _definitionCacheInvalidator;
// Optional (ScadaBridge#100 item 3 / #83): tells the peer central node to drop its caches of
// what the import wrote. Null on hosts without the central cache-invalidation bus.
private readonly ICentralCacheInvalidationPublisher? _cacheInvalidationPublisher;
private readonly ILogger<BundleImporter>? _logger;
/// <summary>
@@ -125,6 +129,7 @@ public sealed class BundleImporter : IBundleImporter
/// <param name="logger">Optional logger.</param>
/// <param name="watermark">Optional: template-graph version watermark, bumped once per apply attempt so the flatten-session caches and the process-wide staleness memo cannot serve pre-import (or rolled-back) readings. Null on hosts that don't register it.</param>
/// <param name="definitionCacheInvalidator">Optional (Gitea #83): drops this node's external-system / database-connection definition cache after a committed import, so imported definitions take effect immediately rather than after the cache TTL. Null on hosts without the External System Gateway.</param>
/// <param name="cacheInvalidationPublisher">Optional (ScadaBridge#100 item 3): tells the peer central node to drop its caches of what a committed import wrote. Null on hosts without the central cache-invalidation bus.</param>
public BundleImporter(
BundleSerializer bundleSerializer,
ManifestValidator manifestValidator,
@@ -147,7 +152,8 @@ public sealed class BundleImporter : IBundleImporter
IScriptArtifactChangeBus? scriptArtifactChangeBus = null,
ILogger<BundleImporter>? logger = null,
ITemplateGraphWatermark? watermark = null,
IExternalDefinitionCacheInvalidator? definitionCacheInvalidator = null)
IExternalDefinitionCacheInvalidator? definitionCacheInvalidator = null,
ICentralCacheInvalidationPublisher? cacheInvalidationPublisher = null)
{
_bundleSerializer = bundleSerializer ?? throw new ArgumentNullException(nameof(bundleSerializer));
_manifestValidator = manifestValidator ?? throw new ArgumentNullException(nameof(manifestValidator));
@@ -170,6 +176,7 @@ public sealed class BundleImporter : IBundleImporter
_scriptArtifactChangeBus = scriptArtifactChangeBus;
_watermark = watermark;
_definitionCacheInvalidator = definitionCacheInvalidator;
_cacheInvalidationPublisher = cacheInvalidationPublisher;
_logger = logger;
}
@@ -2498,18 +2505,20 @@ public sealed class BundleImporter : IBundleImporter
/// served the pre-import definition for up to <c>DefinitionCacheTtl</c>. Invalidates on every
/// committed import rather than only when an external entity was resolved: dependencies can
/// pull definitions in implicitly, imports are rare, and a cleared cache only costs one
/// re-query per name. Never throws (the transaction has already committed).
/// re-query per name. The peer central node is told too (ScadaBridge#100 item 3), since its
/// copy of the cache is just as stale. Never throws (the transaction has already committed).
/// </summary>
private void InvalidateExternalDefinitionCache()
{
if (_definitionCacheInvalidator is null)
if (_definitionCacheInvalidator is null && _cacheInvalidationPublisher is null)
{
return;
}
try
{
_definitionCacheInvalidator.InvalidateAll();
_definitionCacheInvalidator?.InvalidateAll();
_cacheInvalidationPublisher?.Publish(new ExternalDefinitionsInvalidated());
}
catch (Exception ex)
{