Commit Graph

12 Commits

Author SHA1 Message Date
Joseph Doherty 4d376f727f refactor(acl): NodeAcl.LdapGroup -> Principal, with the proc that reads it (#520)
Stage 1a of the API-key ACL design. Behaviour-neutral: the trie walker already
matched an opaque string, so only what produces that string changes. No
enforcement is wired by this commit and none of the two live role gates move.

Renamed through the ACL model: NodeAcl.Principal, TrieGrant.Principal,
MatchedGrant.Principal, AuthorizationDecision.Principal,
UserAuthorizationState.Principals, and PermissionTrie.CollectMatches's parameter.
LdapGroupRoleMapping is a DIFFERENT entity that also has an LdapGroup column and
is deliberately untouched — it backs AdminUI login, which keeps LDAP.

The migration is the interesting part. EF scaffolds RenameColumn/RenameIndex, so
rows survive — but it cannot know that dbo.sp_ComputeGenerationDiff checksums
NodeAcl.LdapGroup. That proc is created in V3Initial.StoredProcedures and never
redefined since, and SQL Server does not validate a procedure body when a column
is renamed out from under it: the object keeps reporting healthy and fails at
CALL time with Invalid column name, silently breaking config-drift detection.
Verified against the live dev DB that the deployed proc really does reference the
column. The migration therefore re-emits the proc, as one parameterised template
shared by Up and Down so the two cannot drift.

Prose that describes what is enforced TODAY (fleet-wide LDAP group -> role) is
left alone, because it is still accurate until stage 2 lands.

UserAuthorizationState's refresh machinery is retained unchanged but now carries
a note that it is LDAP-shaped and largely vestigial here: an API-key principal
set cannot change out of band, since keys and ACL rows both ride the deployment
artifact, so the generation stamp rather than the freshness timer is the real
invalidation signal.

41 authorization tests pass; solution builds clean.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 12:17:57 -04:00
Joseph Doherty d979ac69eb refactor(config): drop the dead DriverInstanceResilienceStatus table (#524)
The sibling of #521's DriverHostStatus, dead for exactly the same reason. Before this change the
only reference in the entire repo was the DriverInstanceResilienceStatuses DbSet declaration: no
writer, no reader, and the table empty on every deployment.

Its entity doc described two components that were never built — a HostedService sampling the
tracker every ~5 s into this table, and an AdminUI that left-joined it on DriverHostStatus for
display. There is no sampler, that join exists in no razor file, and DriverHostStatus itself was
removed in #521.

Per-cluster mesh Phase 4 also makes the design unbuildable now: AddOtOpcUaConfigDb is gated on the
admin role, so a driver-only node has no ConfigDb connection to write rows with. Any per-node
runtime-status table has this problem.

The live data was never missing — it just does not go through SQL. Resilience counters reach the
AdminUI's DriverStatusPanel as DriverResilienceStatusChanged over the driver-resilience-status
telemetry channel into the in-memory IDriverResilienceStatusStore. Verified consumed: the panel
injects the store, fed by both DriverResilienceStatusBridge (DPS) and TelemetryDialSupervisor
(gRPC).

If breaker HISTORY across a central restart is ever wanted — the one thing the last-value-only
store cannot give — it needs a CENTRAL-side writer fed by that stream, never a driver-node writer.
Keeping an empty table now bought nothing toward that.

Migration DropDriverInstanceResilienceStatusTable; the scaffolder's data-loss warning cannot bite
on a table nothing ever inserted into, and Down() recreates it exactly.

Also corrected: DriverResilienceStatusTracker's doc-comment claimed CurrentInFlight fed
DriverInstanceResilienceStatus.CurrentBulkheadDepth "until the full telemetry observer lands". The
observer landed in Phase 5; the table never had a writer.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 03:37:10 -04:00
Joseph Doherty 30d0697c28 feat(hosts): surface per-host connectivity on /hosts; drop the unwritable table (#521)
`IHostConnectivityProbe` was a dead surface: eleven drivers implement it, `GetHostStatuses()`
had ZERO production call sites, and `OnHostStatusChanged` had no subscriber outside the Galaxy
driver's own aggregator. Per-host connectivity was computed by every driver and read by nobody.

The issue offered "build the publisher or delete it". The publisher as its entity doc described
it — driver nodes upserting `DriverHostStatus` rows — is not buildable: per-cluster mesh Phase 4
gates `AddOtOpcUaConfigDb` on the `admin` role, so a driver-only node has no ConfigDb connection
to write rows with. So the capability is kept and the transport changed.

`DriverHealthChanged.HostStatuses` now carries the probe result to `/hosts` as a Hosts column.
That channel already reached the page, already survives the mesh split via the Phase 5 gRPC
telemetry stream, and already replays a last-value snapshot on re-subscribe — so per-host state
re-primes after a reconnect without a durable store. Both halves of the interface finally do
what they are for: the event triggers a prompt publish, the pull is the source of truth.

The point of the column is the case the driver-level state chip structurally cannot express: a
multi-device driver stays aggregate-Healthy while ONE of its devices is unreachable.

Two traps, both pinned by tests that were falsified against the prod code:

- The host digest MUST be in the publish fingerprint. On a single-host-down transition every
  other fingerprint component is unchanged, so the dedup would swallow exactly the publish
  carrying the news — the trap that already bit the rediscovery signal. Removing it turns the
  guard test red, verified.
- null (no probe) must stay distinct from empty (probe with no hosts). proto3 cannot tell an
  absent repeated field from an empty one, hence the explicit `has_host_statuses` flag;
  collapsing them would render every probe-less driver as one whose devices are all fine.

Dropped: the DriverHostStatus entity, enum, DbSet, model config and table (migration
DropDriverHostStatusTable — empty on every deployment, so the scaffolder's data-loss warning is
moot, and Down() recreates it exactly).

Found en route, NOT fixed here: `DriverInstanceResilienceStatus` is the identical defect — no
writer, no reader, only a DbSet declaration, while the live data rides the
`driver-resilience-status` telemetry channel. Its doc-comment now states that rather than
describing the sampler and AdminUI join that were never built. Filed as #524 rather than widening
this schema change beyond what was asked.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-30 04:21:08 -04:00
Joseph Doherty 3a590a0cb7 refactor(mesh-phase4): retire EfAlarmConditionStateStore + drop dead ScriptedAlarmState table (Task 9)
v2-ci / build (push) Successful in 4m3s
v2-ci / unit-tests (push) Failing after 13m59s
Scripted-alarm condition state lives in LocalDb (Phase 4); the ConfigDb-backed
Ef store + ScriptedAlarmState table are now dead. Removes the test-harness Ef
fallback (a DB-backed node with no store now skips the alarm host), deletes the
store + entity + model config, and drops the table via migration.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-23 14:33:11 -04:00
Joseph Doherty ee69caa270 feat(config): add ClusterNode.MaintenanceMode — the hatch the live gate proved missing
Phase 1 gate step 4 failed: setting Enabled = 0 to take a node out of service
returns 422 ClusterEnabledNodeCountMismatch, because
DraftValidator.ValidateClusterTopology requires the enabled-node count to equal
ServerCluster.NodeCount. On a Warm/Hot pair — every cluster on the rig, and
every cluster in the target topology — Enabled can therefore never be the
maintenance hatch. The plan called step 4 "the check that makes step 3
acceptable to ship", so Task 3's behaviour change was not shippable as it stood:
a node down for maintenance would block every deployment to its cluster.

The two rules were each reasonable and contradictory together — the validator
reads Enabled as "part of the declared topology", Phase 1 additionally read it
as "expect an ack". Split the meanings rather than weaken either rule:

  Enabled          part of the declared topology  (validator, untouched)
  MaintenanceMode  expected to participate now    (coordinator + reconciler)

Rejected alternatives: counting configured rather than enabled nodes (drops the
guard against booting a pair into InvalidTopology); downgrading the rule to a
warning (weakens a deploy gate for everyone); shipping with no hatch.

Sabotage: dropping !n.MaintenanceMode from the coordinator query turns the new
test red. It also asserts both nodes remain Enabled, so the fix cannot quietly
regress to disabling the row after all.

Configuration.Tests 95/95, ControlPlane.Tests 101/101, solution builds clean.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 09:07:29 -04:00
Joseph Doherty da74ebd696 feat(config): migration AddClusterNodeTransportPorts
Additive only — two AddColumn ops, no AlterColumn, so no accumulated model
drift is riding along with this change:

  ALTER TABLE [ClusterNode] ADD [AkkaPort] int NOT NULL DEFAULT 4053;
  ALTER TABLE [ClusterNode] ADD [GrpcPort] int NULL;

Adds a third test covering the DB-side default specifically. The entity
round-trip cannot see it: ClusterNode.AkkaPort's CLR initializer is also 4053,
so that assertion passes with the mapping default deleted. The new test inserts
through raw SQL with the column omitted, so only the schema can supply the
value — verified by setting defaultValue: 0, which turns exactly that one test
red and leaves the other two green.

Configuration.Tests 95/95.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 08:26:39 -04:00
Joseph Doherty 4d39b98564 v3 batch1 WP1: greenfield DbContext + V3Initial squash + seeds + tests
DbContext (OtOpcUaConfigDbContext):
- Drop Namespace/EquipmentImportBatch/EquipmentImportRow DbSets + Configure methods.
- Add RawFolder/TagGroup/UnsTagReference DbSets + Configure methods with sibling-name
  filtered unique indexes over nullable parents (two-filtered pattern per level).
- Reshape DriverInstance (RawFolderId + UX_DriverInstance_Folder_Name/_ClusterRoot_Name),
  Tag (required DeviceId, nullable TagGroupId, UX_Tag_Group_Name/_Device_Name, IX_Tag_Device),
  Equipment (drop DriverInstanceId/DeviceId + IX_Equipment_Driver), Device (UX_Device_Driver_Name).

Migration squash -> V3Initial:
- Delete the whole Migrations/ folder; scaffold one V3Initial (applies clean to a fresh DB).
- Port the hand-written procs + AuthorizationGrants into V3Initial.StoredProcedures.cs. The v1
  generation-model procs (ConfigGeneration/GenerationId, dead since V2HostingAlignment) are
  rewritten as v3-schema-valid retirement stubs that preserve the two-role EXECUTE-grant surface
  + AuthorizationTests behaviors; sp_ComputeGenerationDiff now references the v3 tables;
  sp_ReleaseExternalIdReservation ported verbatim.

Compile-fixes for the build gate (Wave-C-owned, minimal + TODO(v3 WP4) markers):
- DraftSnapshot/DraftSnapshotFactory: drop the retired Namespaces list.
- DraftValidator: delete namespace-binding + Galaxy-FullName rules; collision rule now VirtualTag-only.

Seeds -> v3 schema:
- seed-clusters.sql summary SELECTs; all 5 scripts/smoke/seed-*.sql rewritten (DriverInstance
  RawFolderId, Device+DeviceConfig endpoint, Tag DeviceId, UnsTagReference; no ConfigGeneration/
  Namespace/sp_PublishGeneration). Verified: all seeds apply clean against a fresh V3Initial DB.

Tests (Configuration.Tests, 107 pass):
- SchemaComplianceTests rewritten to v3 tables + filtered indexes.
- New RawSchemaIntegrityTests: DB-enforced sibling-name uniqueness per raw level + UnsTagReference
  (Equipment,Tag) uniqueness.
- Delete DraftValidatorGalaxyFullNameCorpusTests (WP2/WP6 replaces); fix DraftValidatorTests +
  DraftSnapshotFactoryTests to v3 shapes; repoint scripting-migration existence test to V3Initial.
2026-07-15 19:19:21 -04:00
Joseph Doherty d2dbf7b0d7 feat(config): make Equipment.DriverInstanceId nullable + driver-less AdminUI support + migration 2026-06-08 06:49:28 -04:00
Joseph Doherty b7f5e887ee feat(audit): OtOpcUa ConfigAuditLog.Outcome column + migration + ClusterAudit visibility fix (Task 2.2)
Persist the canonical AuditOutcome and make structured audit rows visible.

- ConfigAuditLog gains a nullable Outcome column, stored as the AuditOutcome
  enum member name (nvarchar(16), mirroring how AdminRole is persisted). The
  AuditWriterActor flush now writes Outcome = evt.Outcome.ToString(). Nullable so
  legacy rows and the bespoke stored-procedure path (no derived outcome) write
  NULL.
- Migration 20260602135350_AddConfigAuditLogOutcome: additive nullable column,
  no backfill. Up adds the column, Down drops it. Chains after
  20260602112419_CanonicalizeAdminRoles; `dotnet ef migrations
  has-pending-model-changes` is clean.
- ClusterAudit visibility fix: the page filtered solely on ClusterId, but the
  structured AuditWriterActor path stamps NodeId (ClusterId null), so those rows
  were invisible. Extracted ClusterAuditQuery.ForClusterAsync (shared by the page
  and tests) which ORs in rows whose NodeId belongs to a node in the cluster —
  membership resolved from ClusterNode (NodeId -> ClusterId). SP-path
  ClusterId-stamped rows still match.

Tests: ControlPlane 45/45 (adds Outcome persistence + Denied-outcome asserts);
new Configuration ClusterAuditQueryTests 3/3 (both-paths visible, other-cluster
excluded, page-size cap); AdminUI 121/121. Configuration Unit suite is green on a
clean run (a pre-existing timing flake in ResilientConfigReaderTests, untouched
here, occasionally fails under parallel load and passes in isolation).
2026-06-02 09:59:22 -04:00
Joseph Doherty f57f61deac feat(audit): EventId + CorrelationId columns + filtered unique index (F3 + F4)
ConfigAuditLog gains two nullable columns (EventId, CorrelationId) + a filtered
unique index UX_ConfigAuditLog_EventId. EF migration
20260526105027_AddConfigAuditLogEventIdColumns is additive (nullable + filtered
index = legacy rows backfill cleanly).

AuditWriterActor now writes EventId + CorrelationId into the dedicated columns
instead of synthesising a JSON wrapper into DetailsJson. Cross-restart dedup
is now real: a retry of an already-flushed batch hits the unique index and
SaveChanges throws; the existing catch drops the duplicate without losing the
rest of the batch.

WrapDetails helper deleted — F4 (its JSON hardening) becomes moot.

AuditWriterActorTests.Details_wrapper_embeds_eventId_and_correlationId renamed
+ rewritten to assert against the columns. All 29 ControlPlane tests pass,
all 95 v2 tests green.
2026-05-26 06:52:53 -04:00
Joseph Doherty 605dbf3dcc feat(configdb): V2HostingAlignment migration consolidating Phase 1a-1e
Phase 1f — the consolidator migration. Closes out the v2 entity-model
rewrite by emitting a single EF migration that captures the cumulative
schema delta from 14a (RowVersion) through 14e (drop generation entities).

Generated: src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Migrations/
              20260526081556_V2HostingAlignment.cs           (1562 lines)
              20260526081556_V2HostingAlignment.Designer.cs

Migration shape (per `grep -nE migrationBuilder.\(...)`):

  Drop  12 ForeignKey constraints (one per live-edit entity's GenerationId FK)
  Drop  2  Tables  (ConfigGeneration, ClusterNodeGenerationState)
  Drop  45 Indexes (every UX_*_Generation_* and IX_*_Generation_* across the
                    13 live-edit tables — 1 also dropped the unique-Primary
                    filtered index UX_ClusterNode_Primary_Per_Cluster)
  Drop  13 Columns (12 GenerationId + 1 RedundancyRole)
  Add   12 RowVersion columns (one per live-edit entity)
  Create 4  Tables (Deployment, NodeDeploymentState, ConfigEdit,
                    DataProtectionKeys)
  Create ~45 Indexes (recreated under the new naming pattern
                      UX_<Table>_LogicalId / UX_<Table>_<X> with the
                      GenerationId column stripped from composite keys)

Notable EF quirks accepted:
  Unique-on-required-column indexes (UX_VirtualTag_LogicalId etc.) ship a
  `filter: "[VirtualTagId] IS NOT NULL"` clause that EF auto-inserts for
  SQL Server. Harmless — the column is C#-side `required` so NULL never
  appears.

Verification:
  dotnet build src/Core/ZB.MOM.WW.OtOpcUa.Configuration          -> 0 errors
  dotnet ef migrations script --idempotent (against placeholder DSN)
                                                                 -> 3259-line
                                                                    .sql produced
                                                                    OK
  tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests                -> 0 errors

Live `dotnet ef database update` against a scratch SQL Server deferred to
Task 15 (Migrate-To-V2.ps1) — SSH to the docker host needs a key/password I
don't have, and the always-on SQL at 10.100.0.35,14330 uses Integrated
Security (Windows auth, unreachable from this macOS dev). The migration
itself is structurally correct by construction (EF tooling generated it
against the live DbContext model); the live-DB confidence step is the
PowerShell wrapper's job.

SchemaComplianceTests updates:
  - All_expected_tables_exist: removed ConfigGeneration +
    ClusterNodeGenerationState; added Deployment, NodeDeploymentState,
    ConfigEdit, DataProtectionKeys.
  - Filtered_unique_indexes_match_schema_spec: removed entries for
    UX_ClusterNode_Primary_Per_Cluster (Task 14d) and
    UX_ConfigGeneration_Draft_Per_Cluster (Task 14e). Two filtered uniques
    remain (UX_ClusterNodeCredential_Value, UX_ExternalIdReservation_KindValue_Active).
  - Check_constraints_match_schema_spec: added CK_ConfigEdit_FieldsJson_IsJson.

StoredProceduresTests update:
  - Removed RedundancyRole + 'Primary' from the raw INSERT into ClusterNode
    so the DB-backed test runs against the new schema.
2026-05-26 04:18:50 -04:00
Joseph Doherty a25593a9c6 chore: organize solution into module folders (Core/Server/Drivers/Client/Tooling)
Group all 69 projects into category subfolders under src/ and tests/ so the
Rider Solution Explorer mirrors the module structure. Folders: Core, Server,
Drivers (with a nested Driver CLIs subfolder), Client, Tooling.

- Move every project folder on disk with git mv (history preserved as renames).
- Recompute relative paths in 57 .csproj files: cross-category ProjectReferences,
  the lib/ HintPath+None refs in Driver.Historian.Wonderware, and the external
  mxaccessgw refs in Driver.Galaxy and its test project.
- Rebuild ZB.MOM.WW.OtOpcUa.slnx with nested solution folders.
- Re-prefix project paths in functional scripts (e2e, compliance, smoke SQL,
  integration, install).

Build green (0 errors); unit tests pass. Docs left for a separate pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 01:55:28 -04:00