15 Commits

Author	SHA1	Message	Date
Joseph Doherty	8b7ea3213d	fix(rig,docs): backfill ClusterNode.GrpcPort and document the upgrade step (#493) ... `GrpcPort` is a nullable column added by Phase 5. Every insert in the docker-dev seed is INSERT-if-not-exists — the right shape for a re-runnable seed, but it means a new nullable column never reaches rows an earlier version of the file created. On a persisted volume all six rows kept NULL, central found no dial targets, and `TelemetryDial:Mode=Grpc` connected to nothing. (`AkkaPort` escaped this only because it is NOT NULL with a default of 4053.) Seed now backfills `GrpcPort IS NULL AND CreatedBy = 'docker-dev-seed'` to 4056. NULL only — it must not overwrite a port an operator set deliberately on a long-lived dev volume, and NULL is the unambiguous "predates the column" marker. Deliberately NOT doing the suggested EF data migration for real deployments. The correct value is each node's own `Telemetry:GrpcListenPort`, which is per-deployment configuration the database cannot derive; a migration could only invent one, and a WRONG dial target is worse than the NULL the dialer already skips explicitly with a throttled warning. The rig can be backfilled because there the port is fixed by docker-compose.yml and therefore actually knowable. `docs/Telemetry.md` gains an upgrade section stating the prerequisite, the symptoms (the throttled skip log, LISTEN with no ESTABLISHED, the pill never going live) and the UPDATE to run before flipping any node to Grpc — plus why there is no migration.	2026-07-26 10:29:51 -04:00
Joseph Doherty	ffb75b7854	chore(mesh-phase5): rig telemetry ports + shared key + ClusterNode.GrpcPort seed ... Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-23 16:39:29 -04:00
Joseph Doherty	57e1d01766	docs(mesh): rig seed + Phase 1 documentation ... - docker-dev seed: AkkaPort = 4053 on all six ClusterNode rows, so a freshly seeded rig matches a migrated one. GrpcPort left null. - config-db-schema.md: both columns, why AkkaPort is NOT NULL/4053 and GrpcPort is nullable, the unenforced duplication + its reconciler, and Enabled's new second meaning as the deploy path's expected-ack set. - Configuration.md: Cluster:Port / PublicHostname now flag that they are stored twice, with the "update the row too" instruction and why the drift is silent. - design doc §7: Phase 1 marked done, plus a "Phase 1 as shipped" note recording both deviations rather than leaving the sketch reading as what happened. - program plan: Phase 1 marked done; AdminUI node edit explicitly deferred. - CLAUDE.md: the deploy-path behaviour change and its three consequences. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW	2026-07-22 08:38:45 -04:00
Joseph Doherty	cde01e3073	docker-dev: seed a Modbus driver in MAIN (10.100.0.35:5020) for the v3 Batch 1 live gate	2026-07-15 22:03:18 -04:00
Joseph Doherty	4d39b98564	v3 batch1 WP1: greenfield DbContext + V3Initial squash + seeds + tests ... DbContext (OtOpcUaConfigDbContext): - Drop Namespace/EquipmentImportBatch/EquipmentImportRow DbSets + Configure methods. - Add RawFolder/TagGroup/UnsTagReference DbSets + Configure methods with sibling-name filtered unique indexes over nullable parents (two-filtered pattern per level). - Reshape DriverInstance (RawFolderId + UX_DriverInstance_Folder_Name/_ClusterRoot_Name), Tag (required DeviceId, nullable TagGroupId, UX_Tag_Group_Name/_Device_Name, IX_Tag_Device), Equipment (drop DriverInstanceId/DeviceId + IX_Equipment_Driver), Device (UX_Device_Driver_Name). Migration squash -> V3Initial: - Delete the whole Migrations/ folder; scaffold one V3Initial (applies clean to a fresh DB). - Port the hand-written procs + AuthorizationGrants into V3Initial.StoredProcedures.cs. The v1 generation-model procs (ConfigGeneration/GenerationId, dead since V2HostingAlignment) are rewritten as v3-schema-valid retirement stubs that preserve the two-role EXECUTE-grant surface + AuthorizationTests behaviors; sp_ComputeGenerationDiff now references the v3 tables; sp_ReleaseExternalIdReservation ported verbatim. Compile-fixes for the build gate (Wave-C-owned, minimal + TODO(v3 WP4) markers): - DraftSnapshot/DraftSnapshotFactory: drop the retired Namespaces list. - DraftValidator: delete namespace-binding + Galaxy-FullName rules; collision rule now VirtualTag-only. Seeds -> v3 schema: - seed-clusters.sql summary SELECTs; all 5 scripts/smoke/seed-*.sql rewritten (DriverInstance RawFolderId, Device+DeviceConfig endpoint, Tag DeviceId, UnsTagReference; no ConfigGeneration/ Namespace/sp_PublishGeneration). Verified: all seeds apply clean against a fresh V3Initial DB. Tests (Configuration.Tests, 107 pass): - SchemaComplianceTests rewritten to v3 tables + filtered indexes. - New RawSchemaIntegrityTests: DB-enforced sibling-name uniqueness per raw level + UnsTagReference (Equipment,Tag) uniqueness. - Delete DraftValidatorGalaxyFullNameCorpusTests (WP2/WP6 replaces); fix DraftValidatorTests + DraftSnapshotFactoryTests to v3 shapes; repoint scripting-migration existence test to V3Initial.	2026-07-15 19:19:21 -04:00
Joseph Doherty	4bd7180e7f	fix(docker-dev): stop seeding retired SystemPlatform namespace ... The seed re-inserted a Namespace with Kind='SystemPlatform' (+ a GalaxyMxGateway driver + 3 mirror tags), but that NamespaceKind member was removed when Galaxy became Equipment-kind (migration CleanupSystemPlatformNamespaces). cluster-seed runs after the migrator, so a fresh down -v/up re-introduced a Kind the current code can't EF-materialize — 500ing /deployments and failing every publish (ConfigComposer reads db.Namespaces). Remove the obsolete inserts; author an Equipment-kind Galaxy driver via the UI if a fixture is needed.	2026-06-15 12:17:02 -04:00
Joseph Doherty	261419870a	fix(docker-dev): make cluster-seed Galaxy Tag insert idempotent on UX_Tag_FolderPath ... The three TestMachine_001 folder-path tag inserts were guarded by TagId, but the violated constraint is UX_Tag_FolderPath — a unique filtered index on (DriverInstanceId, FolderPath, Name) WHERE EquipmentId IS NULL. The Galaxy driver auto-materialises these same folder-path tags at runtime under its own generated TagIds (nw-mirror-*), so on a re-run the TagId guard found nothing, the insert fired, and it collided on UX_Tag_FolderPath — cluster-seed exited 1. Guard each insert on the index's natural key instead, so the row is skipped whether it came from a prior seed or the driver's runtime mirror. Verified by re-running cluster-seed against the populated dev volume (which holds nw-mirror-testmachine_001-testalarm00{1,2,3}): exit 0, no UX_Tag_FolderPath violation.	2026-06-09 08:29:47 -04:00
Joseph Doherty	b0a62a9f3b	fix(docker-dev): self-bootstrap schema via one-shot migrator (fixes fresh-volume quirks) ... Adds a 'migrator' Dockerfile stage + Compose service that runs 'dotnet ef database update' once on bring-up, so a fresh SQL volume gets the schema with no operator step (quirk 1). cluster-seed + every host node depend on it via service_completed_successfully, so the seed never races an in-progress migration (quirk 2). Host build pinned to target: runtime (the migrator is now the last stage). entrypoint + README updated; the manual 'dotnet ef' first-time step is gone. Verified: down -v + up --build self-bootstraps (migrator+seed exit 0, 6 nodes up), deploy Sealed 6/6.	2026-06-07 08:20:56 -04:00
Joseph Doherty	05471dc36c	feat(docker-dev): seed MAIN ClusterNodes as central-1/central-2	2026-06-07 03:09:16 -04:00
Joseph Doherty	1d7028c2f9	feat(auth): un-stub docker-dev onto shared GLAuth 10.100.0.35 + seed OtOpcUa-* role mappings ... The 6 admin/site host containers drop DevStubMode and bind the shared dev GLAuth (scadaproj/infra/glauth/, dc=zb,dc=local) via cn=serviceaccount. seed-clusters.sql seeds system-wide LdapGroupRoleMapping rows OtOpcUa-Admins->Administrator, OtOpcUa-Designers->Designer, OtOpcUa-Viewers->Viewer (bare-RDN group keys, matching the shared lib's ToGroupShortName). Verified: multi-role -> Viewer+Designer+ Administrator at :9200 via real LDAP.	2026-06-04 16:06:43 -04:00
Joseph Doherty	7dfbca6469	feat(opcua): materialise SystemPlatform tags (Galaxy) as OPC UA variables ... Closes the gap where Tag rows with EquipmentId=NULL + Namespace.Kind=SystemPlatform (Galaxy hierarchy) existed in ConfigDb but were never surfaced in the OPC UA address space. Now they materialise as Variable nodes under a folder named for their FolderPath, browseable through any OPC UA client. Layers touched: - IOpcUaAddressSpaceSink: new EnsureVariable(nodeId, parentFolderId, displayName, dataType) signature on the sink interface, NullSink, DeferredSink, SdkSink. - OtOpcUaNodeManager.EnsureVariable: creates a BaseDataVariableState parented under the named folder (or root), initial Value=null + StatusCode=BadWaitingForInitialData; resolves Tag.DataType strings to the matching OPC UA built-in NodeId. Idempotent. - Phase7CompositionResult: new GalaxyTags collection of GalaxyTagPlan records carrying (TagId, DriverInstanceId, FolderPath, DisplayName, DataType, MxAccessRef). Constructor overloads keep existing call sites compiling. - Phase7Composer.Compose: now takes Tag + Namespace inputs, filters for SystemPlatform-namespace tags with EquipmentId=NULL, emits GalaxyTagPlan rows with MXAccess ref "FolderPath.Name". - Phase7Plan: new AddedGalaxyTags / RemovedGalaxyTags / ChangedGalaxyTags collections + GalaxyTagDelta record; IsEmpty + needsRebuild updated. - Phase7Planner.Compute: diffs GalaxyTags by TagId via existing DiffById helper. - DeploymentArtifact.ParseComposition: reads the Tags + Namespaces + DriverInstances arrays the ConfigComposer already emits, applies the same SystemPlatform filter, returns the same GalaxyTagPlan list as the composer so artifact-side and compose-side plans agree. - Phase7Applier: new MaterialiseGalaxyTags pass that ensures one folder per distinct FolderPath then one Variable per tag. NodeId for the variable is "<FolderPath>.<Name>" matching the MXAccess ref so the future Galaxy SubscribeBulk wiring can address them directly. - OpcUaPublishActor.RebuildAddressSpace: invokes MaterialiseGalaxyTags after MaterialiseHierarchy. _lastApplied initialiser updated for the new ctor. - seed-clusters.sql: pre-existing TestMachine_001.TestAlarm001..003 rows needed no change — the composer/applier now picks them up automatically. Verified end-to-end via docker-dev: deploy click → driver-a logs "Phase7Applier: Galaxy tags materialised (tags=3, folders=1)" → OPC UA Client CLI browses the three Variable nodes under TestMachine_001 folder. Reads return BadWaitingForInitialData status (expected — Galaxy driver's SubscribeBulk wiring to push values into the nodes is the remaining follow-up).	2026-05-26 15:43:22 -04:00
Joseph Doherty	44b8a9c7ff	fix(deploy): ClusterNode NodeId uses host:port + Traefik sticky cookie ... Two bring-up issues found while clicking through the operator Deploy flow on the docker-dev stack: - ConfigPublishCoordinator computes expected-ack NodeIds from Akka.Cluster.State.Members as "{host}:{port}" (e.g. "driver-a:4053") to match ClusterRoleInfo's NodeId derivation. The seed had been using the bare service name ("driver-a"), so NodeDeploymentState INSERT hit FK violation 547 on NodeDeploymentState.NodeId → ClusterNode.NodeId. Seed now writes the full host:port form for every ClusterNode row. - Blazor Server uses SignalR (WebSocket upgrade after the initial GET). Without sticky sessions, Traefik round-robins admin-a/admin-b and the WebSocket upgrade lands on the wrong backend, returning "No Connection with that ID: Status code '404'" so @onclick handlers never fire on the client. Added sticky.cookie (otopcua_lb, SameSite=Lax) to all three Traefik service loadBalancers so each session pins to one node. Verified end-to-end: clicked "Deploy current configuration" on /deployments → Deployment row sealed in ~70ms → driver-a + driver-b spawn GalaxyMxGateway driver (stub=False) → GalaxyDriver connects to http://10.100.0.48:5120 with the seeded ApiKeySecretRef=env:GALAXY_MXGW_API_KEY.	2026-05-26 15:10:11 -04:00
Joseph Doherty	60beb9128e	feat(deploy,runtime): wire mxaccessgw connection — endpoint, key, seed row ... User confirmed the mxaccessgw client (Galaxy driver) doesn't need Windows — only the gateway worker has that constraint. This wires the Galaxy driver into the docker-dev fleet: - docker-compose.yml: GALAXY_MXGW_API_KEY env var on every host service (admin nodes harmlessly ignore it; driver-role nodes pick it up when the seeded DriverInstance resolves ApiKeySecretRef=env:GALAXY_MXGW_API_KEY). Default value matches the key the operator provided; override via shell env (GALAXY_MXGW_API_KEY=... docker compose up -d) to rotate without editing compose. - seed-clusters.sql: now creates a SystemPlatform Namespace (MAIN-galaxy, urn:zb:docker-dev:galaxy) plus a GalaxyMxGateway DriverInstance (MAIN-galaxy-mxgw) in the MAIN cluster pointing at http://10.100.0.48:5120 with UseTls=false. Idempotent via IF NOT EXISTS. - DriverInstanceActor.ShouldStub: clarified the doc comment — only the legacy "Galaxy" type name and "Historian.Wonderware" are Windows-only; the v2 "GalaxyMxGateway" driver is .NET 10 cross-platform (gRPC to an external gateway) and is NOT stubbed. - README: documents the final operator step — sign in, click "Deploy current configuration" on /deployments to materialise the seeded Galaxy driver into a running gRPC connection. Raw DriverInstance rows don't spawn drivers on their own; the v2 lifecycle requires a sealed Deployment first.	2026-05-26 14:58:02 -04:00
Joseph Doherty	ed1c17bc7b	fix(deploy,host): docker-dev bring-up — anon health probes, robust seeder ... Two fixes surfaced while bringing up the docker-dev stack end-to-end: - HealthEndpoints.MapOtOpcUaHealth now calls .AllowAnonymous() on /health/ready, /health/active, /healthz. Without it the AddOtOpcUaAuth fallback policy 401s every probe and Traefik marks every backend unhealthy → all three cluster routes return 503. - cluster-seed entrypoint no longer attempts to apply Migrate-To-V2.sql via sqlcmd. The EF-generated idempotent script puts CREATE PROCEDURE inside IF NOT EXISTS BEGIN ... END blocks (procs must be first in their batch), so sqlcmd fails with "Must declare the scalar variable @FromGenerationId". EF's own runner handles this; sqlcmd doesn't. The seed now just waits for the schema and applies row inserts. Migrations remain the operator's job: dotnet ef database update --project src/Core/.../Configuration \ --startup-project src/Server/.../Host Also: LDAP service removed (bitnami/openldap:2.6 image retired, legacy tag crashes mid-setup with exit 68); every host now runs with Authentication__Ldap__DevStubMode=true. Bumped LDAP+Traefik dashboard host ports to avoid collisions with the sister scadalink dev stack (3893→3894, 8080→8089). Confirmed working end-to-end: all three Traefik routes return HTTP 200, cluster-seed populates ServerCluster (MAIN/SITE-A/SITE-B) + ClusterNode (driver-a/b, site-a-1/2, site-b-1/2) rows on first boot.	2026-05-26 14:37:01 -04:00
Joseph Doherty	f02071c9a2	feat(deploy): bake the ServerCluster/ClusterNode seed into docker-compose ... Adds a one-shot cluster-seed service to docker-dev/docker-compose.yml that pre-populates the three Akka clusters' scope rows in the shared OtOpcUa ConfigDb so operators don't have to click through /clusters + /hosts on every fresh bring-up. Seed contents: ServerCluster MAIN (Warm/2), SITE-A (Warm/2), SITE-B (Warm/2) ClusterNode driver-a + driver-b → MAIN site-a-1 + site-a-2 → SITE-A site-b-1 + site-b-2 → SITE-B NodeCount + RedundancyMode honour the CK_ServerCluster check constraint. ApplicationUri follows the urn:OtOpcUa:<NodeId> convention; uniqueness across the fleet satisfies UX_ClusterNode_ApplicationUri. Mechanism: - docker-dev/seed/seed-clusters.sql — idempotent INSERTs (IF NOT EXISTS guards on every row). - docker-dev/seed/entrypoint.sh — bash wrapper that waits for SQL to accept connections, then polls until dbo.ServerCluster exists (the host containers' EF auto-migration creates it on first boot), then applies the SQL script. - cluster-seed service uses mcr.microsoft.com/mssql-tools as the base image (bash + sqlcmd available), restart: "no" so it runs once. Re-running `docker compose up` is safe: the seed exits cleanly on the second run because every INSERT is guarded. Manual re-seed: `docker compose run --rm cluster-seed`.	2026-05-26 14:06:47 -04:00