Central now ALSO listens for the seven site→central control messages over
gRPC, alongside the existing ClusterClient path. Nothing flips to gRPC yet —
sites keep CentralTransport=Akka (T1A.3's job); central simply starts also
accepting.
- CentralControlGrpcService (Communication.Grpc): decodes each RPC onto the
SAME in-process message the ClusterClient path carries, Asks the existing
CentralCommunicationActor (zero handler-logic changes), encodes the reply via
the T1A.1 mapper. Readiness-gated like SiteStreamGrpcServer.SetReady —
Unavailable until AkkaHostedService hands the actor over. Heartbeat stays
fire-and-forget (Tell, always-OK, never gated on readiness). Ingest reuses the
shared SiteStreamGrpcServer.AuditIngestAskTimeout constant. Fault→status
mapping is retry-aware: Unavailable (never dispatched, safe to cross-node
retry) vs DeadlineExceeded/Internal (it ran, do not re-send elsewhere).
- CentralControlAuthInterceptor (Host): a SEPARATE interceptor class, not a
variant constructor on ControlPlaneAuthInterceptor. Central's model is per-site
(verify the Bearer token against the key for the site in the required
x-scadabridge-site header, via ISitePskProvider) where a site verifies its one
own-key — a genuinely different model. Fail-closed on every branch: missing or
blank header, unresolvable key, and mismatched token all → PermissionDenied,
never pass-through. One public constructor only (the explicit-prefix ctor is
internal), pinned by a reflection test — a second public ctor makes
Grpc.AspNetCore's GetFactory() throw per-call and silently disables the gate.
- Explicit Kestrel h2c listener on new option ScadaBridge:Node:CentralGrpcPort
(default 8083, symmetric with sites), mirroring the Site branch. Additive to
central's :5000 HTTP/1 surface, which is untouched — gRPC does NOT go through
Traefik (HTTP/1 only). Registered by type on AddGrpc; service mapped with
MapGrpcService. Port range-validated by NodeOptionsValidator.
- Rig: publish the central gRPC port 9013:8083 / 9014:8083 on both central nodes
so a later task can exercise it.
Tests: CentralControlEndToEndTests (Host.Tests, TestServer + real interceptor +
real service over a stub actor) proves auth positives/negatives are
distinguishable and covers unary + the ingest bridge shapes; the interceptor is
registered BY TYPE, never in DI. CentralControlAuthInterceptorTests pins the
per-site gate + one-public-ctor invariant. CentralControlGrpcServiceTests
(Communication.Tests, TestKit) covers the readiness gate, fire-and-forget
heartbeat, and the DeadlineExceeded-vs-Unavailable status mapping. No active
<Protobuf> item. Communication.Tests (356) + Host.Tests (384) green.
Phase 1A of the ClusterClient→gRPC migration needs a wire contract for the
seven messages SiteCommunicationActor forwards to /user/central-communication.
This lands the contract and its mapper only — hosting (T1A.2) and the site-side
transport seam (T1A.3) follow.
`Protos/central_control.proto` (package scadabridge.centralcontrol.v1, service
CentralControlService) declares SubmitNotification, QueryNotificationStatus,
IngestAuditEvents, IngestCachedTelemetry, ReconcileSite, ReportSiteHealth and
Heartbeat. Note the direction is the inverse of SiteStreamService: here the site
dials and central serves.
Decisions worth recording:
- The two ingest RPCs IMPORT sitestream.proto and reuse AuditEventBatch /
CachedTelemetryBatch / IngestAck rather than redeclaring them. The site
telemetry actor already builds those messages, so a second copy would fork one
wire contract into two kept in lockstep by hand. ForwardState / IngestedAtUtc
stay off-wire exactly as they are today.
- Heartbeat replies google.protobuf.Empty — it is fire-and-forget and must never
surface a fault onto the heartbeat timer path.
- The three NULLABLE SiteHealthReport collections travel inside single-field
wrapper messages (ConnectionEndpointMapDto / TagQualityMapDto /
NodeStatusListDto). proto3 cannot express presence on repeated/map fields, but
null and empty genuinely differ here — SiteHealthCollector emits
`ClusterNodes: _clusterNodes?.ToList()` and the central health surface reads
null as "not reported", not as "reported empty". Same reasoning drives the
BoolValue/Int64Value/DoubleValue wrappers on LocalDbReplicationConnected,
LocalDbOplogBacklog and the two age gauges, whose docs are explicit that null
is not zero/false.
- ConnectionHealthEnum reserves 0 for UNSPECIFIED instead of mapping Connected
onto it, and the decoder resolves anything unknown to ConnectionHealth.Error.
An unrecognised connection state must not render as "healthy".
- Guid? execution ids travel as "D" strings with empty meaning null; a malformed
non-empty value throws rather than being laundered into "no correlation".
- DateTimeOffset normalizes to a UTC instant (a protobuf Timestamp has no
offset). Lossless in practice — every producer stamps UTC — and documented +
asserted rather than left implicit.
SiteCallDtoMapper gains a ToDto(SiteCall) overload. Its doc comment previously
asserted such a method "would be dead code"; that held only while ClusterClient
was the sole path from IngestCachedTelemetryCommand (which carries SiteCall, not
SiteCallOperational) to central. Comment corrected alongside.
Golden tests round-trip every message through a real protobuf encode/decode —
DTO → proto → bytes → proto → DTO — with a fully-populated case and a
null/empty/minimal case for every optional member. Verified to have teeth by
mutation: dropping a scalar, collapsing an empty nullable collection to absent,
and nulling a gauge each fail a test.
Codegen stays CHECKED IN under CentralControlGrpc/ (protoc segfaults in the
linux_arm64 Docker image); no active <Protobuf> item is committed.
docker/regen-proto.sh is generalized to `regen-proto.sh [sitestream|
centralcontrol|all]` — it now injects the ItemGroup rather than unwrapping a
comment, so it no longer depends on there being exactly one Protobuf line, and
it restores the csproj verbatim on every exit path.
Phase 0 of the ClusterClient→gRPC migration
(docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Standalone hardening: it
closes a gap that exists today and is a precondition for moving command/control
onto gRPC in later phases.
T0.1 — delete the ManagementActor ClusterClientReceptionist registration.
It was built for an out-of-cluster CLI that was never written: the shipped CLI
speaks HTTP Basic to /management, which asks the actor in-process through
ManagementActorHolder. Nothing in the repo ever sent to /user/management. The
actor still runs there; only the cross-boundary advertisement is gone. Six
documents claimed the CLI used ClusterClient — including the CLI's own README
"Architecture Notes" — and are corrected here rather than left to rot.
T0.2 — record, do not port, the dead integration-routing path.
IntegrationCallRequest is unwired at BOTH ends: RouteIntegrationCallAsync has
zero callers anywhere, and RegisterLocalHandler(Integration, …) appears only in
a test, so production always answers "Integration handler not available". It is
excluded from the gRPC contract (28 of 29 commands migrate) rather than
enshrined on an additive-only wire format, and deleting it during a
transport migration would mix a behavioural change into a change whose whole
value is that behaviour is identical. See
docs/known-issues/2026-07-22-integration-call-routing-is-dead-code.md.
T0.3 — preshared-key authentication on SiteStreamService.
The service shipped with no auth at all: plaintext h2c, no interceptor, so
anything that could reach a site node's :8083 could open a live data stream or
read audit rows back via PullAuditEvents/PullSiteCalls. ControlPlaneAuthInterceptor
now gates /sitestream.SiteStreamService/ — modeled on LocalDbSyncAuthInterceptor
(constant-time compare, fail-closed, PermissionDenied) but gating a SET of
service prefixes so phases 1A/1B add services rather than interceptors. LocalDb
sync keeps its own separate key: it authenticates the pair partner, not central,
and collapsing the two would make a site's central-facing key also admit writes
into its database.
Keys are per site (SB-GRPC-PSK-<siteId>), never fleet-wide, so a compromised
site yields only its own. Central attaches them through ControlPlaneCredentials,
which binds CallCredentials to the channel — covering unary and streaming
uniformly, and letting the key resolve asynchronously, which a client
interceptor could not do without blocking. All three central→site channel
creation sites go through it (SiteStreamGrpcClient and both audit pull invokers);
the pull invokers' channel caches are re-keyed by (site, endpoint) because
credentials are per-site and bound to the channel.
Two decisions beyond the plan:
* StartupValidator now requires GrpcPsk on Site nodes. The plan specified only
the runtime gate, but fail-closed with no boot check produces a node that
joins, answers heartbeats and reports healthy while refusing every stream,
audit pull and telemetry ingest — silent and total. Same reasoning as the
existing inbound API-key pepper rule.
* Added Communication:SitePsks as a central-side key map. The plan assumed
central would read the store, seeded via a dev KEK; the docker rig
deliberately boots with no master key, so store-only resolution would leave
it unable to dial its own sites. The store stays primary — it is the only
source that can serve a site added at runtime — with the map covering
key-less hosts and one-off pins. Neither source falling back to
"unauthenticated" is the invariant.
T0.4 — dev keys on both rigs and tests.
34 tests. The seven that matter most exercise a real in-process gRPC stack over
TestServer: the unit tests on either side of the wire would both stay green if
the halves disagreed, and gRPC refuses call credentials on a plaintext channel
by default — the UnsafeUseInsecureChannelCallCredentials opt-in is only provable
by making a real call. They confirm correct key passes on unary AND streaming,
wrong key and no-credentials both get PermissionDenied, and an unresolvable key
fails the call with nothing reaching the service.
OPERATIONAL: a site node upgraded to this build without a key will not boot.
That includes the gitignored deploy/wonder-app-vd03/ overlay.
CentralFailoverControl lives in Components/Health/ (matching AuditKpiTiles /
SiteCallKpiTiles) rather than inline in Health.razor, so it is testable without
standing up the whole dashboard's DI graph.
- Admin-gated via AuthorizeView + RequireAdmin. The Health page itself is
intentionally all-roles, so the gate belongs on the control, not the page.
- Disabled with an explanatory title when the pair has no online standby;
the authoritative guard remains server-side against live cluster membership.
- Confirmation dialog (IDialogService, the page idiom) warns that singletons
hand over, in-flight work on the active node is interrupted, and THIS PAGE
will disconnect and reconnect against the new active node -- Traefik routes
the UI to the node being restarted, so a working failover otherwise reads as
a crash the admin caused.
- A refused failover (service returns null) surfaces the refusal; the UI never
reports a failover that did not happen.
7 bUnit tests. Two harness requirements that bit first: AuthorizeView needs a
cascading AuthenticationState (the app supplies it from the layout), and
BunitContext pre-registers a placeholder IAuthorizationService that throws on
policy evaluation -- both handled the same way NavMenuTests documents.
Runbook paragraphs added to Component-ClusterInfrastructure.md (new Manual
Failover section) and docker/README.md.
Every node now lists ITSELF as seed-nodes[0] and its partner second. Akka runs
FirstSeedNodeProcess -- the only bootstrap path that can form a NEW cluster when
no peer answers InitJoin -- exclusively for seed-nodes[0]; every other node runs
JoinSeedNodeProcess and retries InitJoin forever. That is why a lone cold-starting
central-b never came Up (the "registered outage gap"), and self-first ordering
closes it using Akka's own protocol.
- 6 node appsettings swapped (the *-node-b configs; the -a nodes were already
self-first). All 14 shipped node configs now satisfy the invariant.
- StartupValidator enforces it at boot, comparing host AND port -- the invariant
fails silently when broken, so it is enforced loudly. NOTE: the gitignored
deploy/wonder-app-vd03/ overlay must be reordered before its next deploy or
that node will refuse to boot.
- SelfFirstSeedBootstrapTests: real in-process clusters at production
failure-detection timings, incl. a falsifiability control proving the OLD
peer-first ordering never forms.
Rejected alternative (implemented, measured, discarded): an external self-form
timer calling Cluster.Join(SelfAddress) after a window. It sits outside Akka's
join handshake and so cannot tell "no seed answered" from "a seed answered and
the join is in flight". On a routine standby restart the peer is alive but the
join stalls behind removal of the node's own stale incarnation; a Join(self)
during TryingToJoin abandons the in-flight join and forms a second cluster at
the same address -- still split after 90s. Docs that claimed self-first ordering
was unsafe for simultaneous cold start are corrected: while mutually reachable
the InitJoin handshake converges them to one cluster (measured).
Two-node keep-oldest could NEVER survive a crash of the oldest/active node:
Akka.NET 1.5.62 KeepOldest.OldestDecision only lets down-if-alone rescue a
side with >= 2 members, so the 1-vs-1 survivor takes DownReachable and downs
ITSELF — proven live on the rig ('SBR took decision ... including myself')
before this change. static-quorum(1) is worse (IsTooManyMembers -> DownAll);
keep-majority just re-keys the fatal crash to the lowest address.
SplitBrainResolverStrategy gains 'auto-down' (new default): BuildHocon emits
Akka's AutoDowning provider with auto-down-unreachable-after = StableAfter.
The leader among the REACHABLE members downs the unreachable peer, so the
survivor takes over singletons and /health/active in ~25s regardless of which
node died. Accepted trade (explicit owner decision): a real network partition
runs dual-active until an operator restarts one side. keep-oldest remains
supported; DownIfAlone validation is now scoped to it.
Live drill on the rebuilt rig: active-crash TAKEOVER in 28s (victim still
down; all 7 singletons Younger->Oldest), standby-crash removal 27s with 0
routing blips; victims rejoin as standby in 2s. New real-cluster tests pin
both directions (SbrFailoverTests.AutoDown_*); TwoNodeClusterFixture gains a
strategy knob. All 16 appsettings flipped (src, docker, docker-env2, and the
gitignored wonder-app-vd03 overlay on disk — owner must sync to the host).
Docs: decision record docs/plans/2026-07-21-auto-down-availability-decision.md,
Component-ClusterInfrastructure downing section rewritten, drill + README
reworked (active mode now asserts takeover), deferred-work SBR row resolved.
MaxBatchSize 500 -> 16. The default is a ROW count, not a byte budget, so the
batch size in bytes is set by the widest replicated column — config_json, which
Task 1 measured at up to ~60-70 KB in production. 70 KB x 500 is ~35 MB against
gRPC's 4 MB default receive limit; 16 keeps a worst-case batch near 1.1 MB.
MaxOplogRows 1,000,000 -> 250,000 and MaxOplogAge 7d -> 2d, sized from the
soak's 0.80 sf_messages rows/sec (~69k rows/day). Tighter than default is
correct here because exceeding a cap is not data loss: the oplog prunes to the
ceiling and sets needs_snapshot, so the peer catches up by snapshot resync
instead of incrementally. That trades a rare full resync for a bounded file.
site-b and site-c stay unreplicated, so the default-OFF posture is still proven
side by side on one rig.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
LocalDb Phase 2 deleted the bespoke replicators, so three config keys changed
meaning or died outright:
- ScadaBridge:StoreAndForward:ReplicationEnabled is fully dead. Deleted the
property, its 10 config entries, and the 5 test references.
- SqliteDbPath / SiteDbPath are now migration-only: they name the legacy files
SiteLocalDbLegacyMigrator drains at boot, not live databases. Both mandatory
rules are relaxed accordingly (StartupValidator's Site-only Require, and the
S&F validator's non-empty rule) — an absent value now means "nothing to
migrate", so an already-migrated node can drop the key. DatabaseOptions-
Validator still rejects a present-but-blank value.
- SiteRuntime:ConfigFetchRetryCount's only reader was SiteReplicationActor.
Deleted with its validator rule.
The two path keys stay present in every config, now with a comment explaining
why: removing them would strand un-migrated data on a node that has not yet
started once.
Both relaxations are pinned by the inverse of the test they replace
(Site_MissingSiteDbPath_IsAccepted..., EmptySqliteDbPath_IsAccepted...), each
verified to fail with the old rule restored.
Note: deploy/wonder-app-vd03/appsettings.Site.json is under a gitignored
deploy/ tree, so its edit is local-only and must be repeated on the box.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
docker/seed-sites.sh inserted the pre-rename 'Design' / 'Deployment' role
strings, but the canonical vocabulary in Roles.cs is 'Designer' / 'Deployer'.
The mismatch authorized nothing: on a freshly reseeded rig every
Designer/Deployer-gated management command failed UNAUTHORIZED, including
seed-sites.sh's own trailing `deploy artifacts` and reseed.sh's stage 6d
encrypted-secret restore.
Found while standing up the Phase 2 soak rig.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Task 11. site-a-a is the initiator (dials scadabridge-site-a-b:8083, the
existing site gRPC h2c listener - no new port); site-a-b is passive. Both carry
the SAME dev ApiKey, which is load-bearing: LocalDbSyncAuthInterceptor is
fail-closed, so a mismatch does not degrade to unauthenticated replication, it
rejects every stream and the pair silently stops converging.
site-b and site-c stay unreplicated on purpose, so the default-OFF posture is
proven side-by-side on one rig rather than asserted.
Plain dev key matches the rig's existing posture; production uses a
${secret:...} reference through the pre-host expander. Noted in both files.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Task 3. Site nodes now open one ZB.MOM.WW.LocalDb-managed database holding
OperationTracking and site_events as replicated tables.
SiteLocalDbSetup.OnReady applies both schemas and then RegisterReplicated's them.
That order is load-bearing and easy to get silently wrong: capture is
trigger-based CDC, so rows written before registration are never captured or
snapshotted - they would be invisible to the peer forever, with no error anywhere.
Storage ships UNCONDITIONALLY, no feature flag. With no peer configured LocalDb is
just a local SQLite file, so this is behaviour-equivalent to what site nodes do
today. Replication is opt-in and lands in Program.cs (Task 7), which owns the gRPC
host the sync endpoint needs.
Config: LocalDb:Path added to all EIGHT site-node appsettings (the plan said six -
docker-env2's site-x pair exists too), plus the dev template and the wonder-app-vd03
production sample. All ten are required, not optional: LocalDbOptions.Path is
validated with ValidateOnStart, so a site node without it fails fast at startup.
That is the desired posture, and it is what caught ActorPathTests - the one Host
fixture that actually starts a host - whose config now sets the path too.
Note the path choice fixes an existing data-loss bug in passing: site-tracking.db
and site_events.db both defaulted to CWD-relative paths (/app/...), outside the
mounted volume, so they were silently lost on every container recreate. The
consolidated database lives on /app/data.
Tests are container-built against the REAL SiteServiceRegistration.Configure, not a
hand-assembled ServiceCollection - this family has shipped three wiring defects that
only a real graph would catch (inert Secrets replicator 0.2.0, singleton deadlock
0.2.2, Secrets.Ui missing IAuditWriter). They assert the library's own view of what
is registered rather than that we called the right method.
Verified (all after the change):
dotnet build ZB.MOM.WW.ScadaBridge.slnx -> 0 Error(s), 0 Warning(s)
Host.Tests -> 294 passed (5 new, red-first)
SiteRuntime.Tests -> 530 passed
CentralUI.Tests -> 925 passed
Commons.Tests -> 684 passed
Communication.Tests -> 312 passed
SiteEventLogging.Tests-> 70 passed
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Populates ScadaBridge:Communication:CentralFetchBaseUrl in every central
appsettings for the notify-and-fetch deploy flow. An empty value now
causes a fail-fast on deploy; this config prevents that regression.
- docker/: http://scadabridge-traefik (in-network Traefik LB, port 80)
- docker-env2/: http://scadabridge-env2-traefik (env2 Traefik LB, port 80)
- src/Host base: http://localhost:5000 (ASP.NET Core default for single-host dev)
- deploy/wonder-app-vd03: http://localhost:8085 (gitignored; edited in main repo)
RUNBOOK Upgrading section updated with note on this setting.
Wire the M6 KPI History recorder into the central composition path:
- Program.cs: call services.AddKpiHistory(configuration) on the central-only
branch alongside AddNotificationOutbox/AddAuditLog/AddSiteCallAudit.
- AkkaHostedService.cs: register KpiHistoryRecorderActor as a central,
non-role-scoped ClusterSingletonManager + ClusterSingletonProxy + a
PhaseClusterLeave CoordinatedShutdown graceful-stop drain (singleton name
'kpi-history-recorder'), copied/adapted from the audit-log-purge block.
- appsettings.Central.json (Host + docker + docker-env2 central nodes): add a
ScadaBridge:KpiHistory section (SampleInterval 00:01:00, RetentionDays 90,
PurgeInterval 1.00:00:00, DefaultMaxSeriesPoints 200).
KPI history is observability/best-effort and MUST NOT gate readiness: the
recorder is deliberately NOT added to RequiredSingletonsHealthCheck or any
other readiness gate.
Adds a "Dev Disable-Login Flag" subsection to Component-Security.md covering
ScadaBridge:Security:Auth:DisableLogin / User, the AutoLoginAuthenticationHandler
mechanism, and the no-environment-guard / startup-warning production risk.
Ships DisableLogin: false under ScadaBridge → Security → Auth in:
- src/.../Host/appsettings.json (canonical default)
- docker/central-node-a/appsettings.Central.json
- docker/central-node-b/appsettings.Central.json
Also records DL-3 commit SHAs in the plan tasks file.
Both :9000 (docker) and :9100 (docker-env2) central nodes now bind the shared dev
GLAuth (scadaproj/infra/glauth/, dc=zb,dc=local) via the cn=serviceaccount search
account instead of the bundled scadabridge-ldap container (now commented out in
infra/docker-compose.yml, kept for rollback). Verified: multi-role -> all 4 roles
on both clusters with scadabridge-ldap stopped.
The Auth/Config normalization made ScadaBridge:InboundApi:ApiKeyPepper a hard
Central-only startup requirement (>=16 chars), but the local dev composes never
supplied it, so deploy.sh's freshly-built image crash-looped both Central nodes
on ConfigPreflight validation. Add a clearly-marked DEV-ONLY, insecure pepper
inline to each cluster's Central environment (distinct per environment). These
are NOT real secrets — production injects a true per-env secret out-of-band per
docs/operations/inbound-api-key-reissue.md; the inline values exist only so the
local docker / docker-env2 clusters start.
Replace dc=scadabridge,dc=local with dc=zb,dc=local in all dev/test LDAP
references — app config, docker test-cluster node configs (docker/ and
docker-env2/), GLAuth fixture, dev tooling, Host.Tests fixtures,
IntegrationTests factory, and operational test_infra docs. OU structure
(ou=SCADA-Admins,ou=users,etc.) preserved throughout. Email domains
(@scadabridge.local), hostnames, and container names are untouched.
Historical plan docs (2026-05-24-second-environment.md,
2026-05-31-folder-repo-rename-scadabridge-design.md) excluded as
point-in-time records. No synthetic dc=example,dc=com placeholders touched.
Renames the 13 SCADALINK_* runtime env vars → SCADABRIDGE_*, the ScadaLink__
.NET config keys → ScadaBridge__, the stale ScadaLink.Host.exe assembly name
→ ZB.MOM.WW.ScadaBridge.Host.exe, the scadalink_app SQL login → scadabridge_app,
and residual identifiers/comments/docs. Migration records (prior rename
tooling/design, DB-rename helper, this scrub script) carved out.
Adds tools/scrub-scadalink-refs.sh.
Replace the generic 'MxGateway Shared' connection name with a per-site name
(site-a -> 'ScadaBridge Site A', env2 site-x -> 'ScadaBridge Site X') in both
docker/ and docker-env2/ seed scripts.
- DeploymentManagerActor.HandleDeployArtifacts read the Self property inside its
Task.Run lambda (line dispatching ApplyArtifactDataConnectionsToDcl). Self is
backed by the ambient ActorCell, null on a thread-pool thread, so it threw
'no active ActorContext' — surfaced the first time a data connection is
deployed via deploy-artifacts. Capture Self into a local first (as Sender
already was).
- seed-sites.sh: create a shared MxGateway data connection (10.100.0.48:5120)
on each site and deploy artifacts so the DCL establishes them.
- build.sh: nounset-safe empty-array expansion (bash 3.2).
Dockerfile copies nuget.config (so the Gitea feed + source mapping are known in
the restore stage) and accepts optional NUGET_GITEA_USER/PASS build-args for a
private feed; build.sh forwards them from host env vars (MXGW_NUGET_USER/PASS)
so secrets are never committed. Verified the in-container restore resolves the
ZB.MOM.WW.MxGateway.* packages (anonymous feed).
Test instances persistently emit Notify.To("Engineering Alerts").Send;
without the list at central a fresh cutover parks every notification
(observed 42k+ parked in a 3.5-min S&F drain after the rename).
Mirror the seed across docker/seed-sites.sh and docker-env2/seed-sites.sh.
A fresh ScadaBridgeConfig has only the Admin LdapGroupMappings row
(InitialSchema migration ships one row, SecurityConfiguration.HasData
declares four). docker-env2/seed-sites.sh already inserts the missing
three idempotently; docker/seed-sites.sh did not, so multi-role got
Admin only on a primary cutover. Mirror the env2 insert block.
Renames ScadaLink* databases + scadalink_app login on an existing
running scadabridge-mssql container. For users preserving seeded
test data through the rename. Fresh deployments use the already-
updated infra/mssql/setup.sql directly.
- NavMenu: move Import Bundle out of the nested RequireDesign/RequireAdmin
double-gate into the top-level Admin section so an Admin-only user sees it
without needing the Design role; Export Bundle stays in the Design section.
- TransportImport: inject IAuditService + ScadaLinkDbContext; emit a
BundleImportUnlockFailed audit row (best-effort, swallowed on failure) on
every wrong-passphrase attempt in SubmitPassphraseAsync, with attempt
number and error reason in afterState.
- docker central-node-a/b appsettings: add ScadaLink:Transport section with
SourceEnvironment = "docker-cluster" so the importer picks up a non-null
environment name in the audit trail.
- CentralUI.Tests: register IAuditService mock + SQLite in-memory
ScadaLinkDbContext in TransportImportPageTests to satisfy the two new injects.
Adds "NodeName" to the ScadaLink:Node section of each per-node
appsettings:
- central-a, central-b for the two central nodes
- node-a, node-b under each of the three sites (site-a, site-b, site-c)
After this commit + a redeploy, every fresh AuditLog / Notifications /
SiteCalls row gets stamped with the originating node's role name via
INodeIdentityProvider, satisfying the design's SourceNode invariant
end-to-end.
Both central nodes ran on the NotificationOutboxOptions code defaults
(100 / 10s = 600/min) because the mounted per-node appsettings.Central.json
had no ScadaLink:NotificationOutbox section. Add the section with
DispatchBatchSize 1000 + DispatchInterval 5s — measured ~6,000/min after
restart (sweep duration becomes the binding constraint, which is fine:
the no-overlap guard self-regulates). Dev-cluster tuning only.
The cookie SecurePolicy was hard-coded to Always, so the auth cookie was always
marked Secure and the browser never sent it over plain HTTP — making login
impossible on the HTTP-only Docker dev cluster (login succeeded server-side but
every following request was unauthenticated). Add SecurityOptions.RequireHttps-
Cookie (default true — production stays HTTPS-only); when false the cookie uses
SameAsRequest. The docker/ central nodes set it false.
The Dockerfile restore stage copied every .csproj but not
Directory.Packages.props, so Central Package Management (adopted in 9c60592)
had no version source inside the container and 'dotnet restore' failed NU1015
('PackageReference items do not have a version specified'). The image could
not be rebuilt since CPM adoption. Copy the props file above the projects.
Adds a new HiLo alarm trigger type with four configurable setpoints
(LoLo / Lo / Hi / HiHi). Each setpoint carries an optional priority,
deadband (for hysteresis), and operator message. The site runtime emits
AlarmStateChanged with an AlarmLevel field so consumers can differentiate
warning vs critical bands.
Plumbing:
- new AlarmLevel enum + AlarmStateChanged.Level/Message init properties
- AlarmTriggerEditor (Blazor) gets a HiLo render with severity tinting
- AlarmTriggerConfigCodec extracted from the editor for testability
- sitestream.proto carries level + message over gRPC
- SemanticValidator enforces numeric attribute, setpoint ordering,
non-negative deadband
- on-trigger scripts get an Alarm global (Name/Level/Priority/Message)
so notification routing can branch by severity
- per-instance InstanceAlarmOverride entity + EF migration + flattening
step + CLI commands; HiLo overrides merge setpoint-by-setpoint, binary
types whole-replace
- DebugView shows a Level badge + per-band message tooltip
- App.razor auto-reloads on permanent Blazor circuit failure
- docker/regen-proto.sh automates the proto regen workflow (the linux/arm64
protoc segfault means generated files are checked in for now)
The lmxproxy workspace was relocated to deprecated/ in 9dccf8e but the
Dockerfile still tried to COPY lmxproxy/src/ZB.MOM.WW.LmxProxy.Client/,
breaking docker build. Remove the two stale COPY lines.
Docker: include lmxproxy/src/ZB.MOM.WW.LmxProxy.Client in build context
so the project reference resolves during container image build.
CLI: fix set-bindings JSON parsing — use JsonElement.GetString()/GetInt32()
instead of object.ToString() which returned null for deserialized elements.
Switch site host to WebApplicationBuilder with Kestrel HTTP/2 gRPC server,
add GrpcPort/keepalive config, wire SiteStreamManager as ISiteStreamSubscriber,
expose gRPC ports in docker-compose, add site seed script, update all 10
requirement docs + CLAUDE.md + README.md for the new dual-transport architecture.
Add ActiveNodeHealthCheck that returns 200 only on the Akka.NET cluster
leader, enabling Traefik to route traffic to the active central node and
automatically fail over when the leader changes. Also fixes AkkaClusterHealthCheck
to resolve ActorSystem from AkkaHostedService (was always null via DI).
Replace the CLI's Akka.NET ClusterClient transport with a simple HTTP client
targeting a new POST /management endpoint on the Central Host. The endpoint
handles Basic Auth, LDAP authentication, role resolution, and ManagementActor
dispatch in a single round-trip — eliminating the CLI's Akka, LDAP, and
Security dependencies.
Also fixes DCL ReSubscribeAll losing subscriptions on repeated reconnect by
deriving the tag list from _subscriptionsByInstance instead of _subscriptionIds.
Joseph Doherty