diff --git a/CLAUDE.md b/CLAUDE.md index dcbcf29..bc12085 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,17 +5,19 @@ This project contains design documentation for a distributed SCADA system built ## Project Structure - `README.md` — Master index with component table and architecture diagrams. -- `HighLevelReqs.md` — Complete high-level requirements covering all functional areas. -- `Component-*.md` — Individual component design documents (one per component). +- `docs/requirements/HighLevelReqs.md` — Complete high-level requirements covering all functional areas. +- `docs/requirements/Component-*.md` — Individual component design documents (one per component). +- `docs/requirements/lmxproxy_protocol.md` — LmxProxy gRPC protocol specification. +- `docs/test_infra/test_infra.md` — Master test infrastructure doc (OPC UA, LDAP, MS SQL, SMTP, REST API, LmxFakeProxy, Traefik). - `docs/plans/` — Design decision documents from refinement sessions. - `AkkaDotNet/` — Akka.NET reference documentation and best practices notes. -- `test_infra.md` — Master test infrastructure doc (OPC UA, LDAP, MS SQL). - `infra/` — Docker Compose and config files for local test services. - `docker/` — Docker infrastructure for the 8-node cluster topology (2 central + 3 sites). See [`docker/README.md`](docker/README.md) for cluster setup, port allocation, and management commands. ## Document Conventions -- All documents are markdown files in the project root directory. +- Requirements documents (high-level and component-level) live in `docs/requirements/`. +- Test infrastructure documentation lives in `docs/test_infra/`. - Component documents are named `Component-.md` (PascalCase, hyphen-separated). - Each component document follows a consistent structure: Purpose, Location, Responsibilities, detailed design sections, Dependencies, and Interactions. - The README.md component table must stay in sync with actual component documents. When a component is added, removed, or renamed, update the table. diff --git a/README.md b/README.md index 45c1b91..713cdfe 100644 --- a/README.md +++ b/README.md @@ -28,32 +28,32 @@ This document serves as the master index for the SCADA system design. The system ## Document Map ### Requirements -- [HighLevelReqs.md](HighLevelReqs.md) — Complete high-level requirements covering all functional areas. +- [HighLevelReqs.md](docs/requirements/HighLevelReqs.md) — Complete high-level requirements covering all functional areas. ### Component Design Documents | # | Component | Document | Description | |---|-----------|----------|-------------| -| 1 | Template Engine | [Component-TemplateEngine.md](Component-TemplateEngine.md) | Template modeling, inheritance, composition, path-qualified member addressing, override granularity, locking, alarms, flattening, semantic validation, revision hashing, and diff calculation. | -| 2 | Deployment Manager | [Component-DeploymentManager.md](Component-DeploymentManager.md) | Central-side deployment pipeline with deployment ID/idempotency, per-instance operation lock, state transition matrix, all-or-nothing site apply, system-wide artifact deployment with per-site status. | -| 3 | Site Runtime | [Component-SiteRuntime.md](Component-SiteRuntime.md) | Site-side actor hierarchy with explicit supervision strategies, staggered startup, script trust model (constrained APIs), Tell/Ask conventions, concurrency serialization, and site-wide Akka stream with per-subscriber backpressure. | -| 4 | Data Connection Layer | [Component-DataConnectionLayer.md](Component-DataConnectionLayer.md) | Common data connection interface (OPC UA, custom), Become/Stash connection actor model, auto-reconnect, immediate bad quality on disconnect, transparent re-subscribe, synchronous write failures, tag path resolution retry. | -| 5 | Central–Site Communication | [Component-Communication.md](Component-Communication.md) | Akka.NET remoting/cluster topology, 8 message patterns with per-pattern timeouts, application-level correlation IDs, transport heartbeat config, message ordering, connection failure behavior. | -| 6 | Store-and-Forward Engine | [Component-StoreAndForward.md](Component-StoreAndForward.md) | Buffering (transient failures only), fixed-interval retry, parking, async best-effort replication, SQLite persistence at sites. | -| 7 | External System Gateway | [Component-ExternalSystemGateway.md](Component-ExternalSystemGateway.md) | HTTP/REST + JSON, API key/Basic Auth, per-system timeout, dual call modes (Call/CachedCall), transient/permanent error classification, dedicated blocking I/O dispatcher, ADO.NET connection pooling. | -| 8 | Notification Service | [Component-NotificationService.md](Component-NotificationService.md) | SMTP with OAuth2 (M365) or Basic Auth, BCC delivery, plain text, transient/permanent SMTP error classification, store-and-forward integration. | -| 9 | Central UI | [Component-CentralUI.md](Component-CentralUI.md) | Blazor Server with SignalR real-time push, load balancer failover with JWT, all management workflows. | -| 10 | Security & Auth | [Component-Security.md](Component-Security.md) | Direct LDAP bind (LDAPS/StartTLS), JWT sessions (HMAC-SHA256, 15-min refresh, 30-min idle), role-based authorization, site-scoped permissions. | -| 11 | Health Monitoring | [Component-HealthMonitoring.md](Component-HealthMonitoring.md) | 30s report interval, 60s offline threshold, monotonic sequence numbers, raw error counts, tag resolution counts, dead letter monitoring. | -| 12 | Site Event Logging | [Component-SiteEventLogging.md](Component-SiteEventLogging.md) | SQLite storage, 30-day retention + 1GB cap, daily purge, paginated remote queries with keyword search. | -| 13 | Cluster Infrastructure | [Component-ClusterInfrastructure.md](Component-ClusterInfrastructure.md) | Akka.NET cluster, keep-oldest SBR with down-if-alone, min-nr-of-members=1, 2s/10s/15s failure detection, CoordinatedShutdown, automatic dual-node recovery. | -| 14 | Inbound API | [Component-InboundAPI.md](Component-InboundAPI.md) | POST /api/{methodName}, X-API-Key header, flat JSON, extended type system (Object/List), script-based implementations, failures-only logging. | -| 15 | Host | [Component-Host.md](Component-Host.md) | Single deployable binary, role-based component registration, per-component config binding (Options pattern), readiness gating, dead letter monitoring, Akka.NET bootstrap, ASP.NET Core hosting for central. | -| 16 | Commons | [Component-Commons.md](Component-Commons.md) | Namespace/folder convention (Types/Interfaces/Entities/Messages), shared data types, POCOs, repository interfaces, message contracts with additive-only versioning, UTC timestamp convention. | -| 17 | Configuration Database | [Component-ConfigurationDatabase.md](Component-ConfigurationDatabase.md) | EF Core data access, per-component repositories, unit-of-work, optimistic concurrency on deployment status, audit logging (IAuditService), migration management. | -| 18 | Management Service | [Component-ManagementService.md](Component-ManagementService.md) | Akka.NET ManagementActor on central, ClusterClientReceptionist registration, programmatic access to all admin operations, CLI interface. | -| 19 | CLI | [Component-CLI.md](Component-CLI.md) | Standalone command-line tool, System.CommandLine, HTTP transport via Management API, JSON/table output, mirrors all Management Service operations. | -| 20 | Traefik Proxy | [Component-TraefikProxy.md](Component-TraefikProxy.md) | Reverse proxy/load balancer fronting central cluster, active node routing via `/health/active`, automatic failover. | +| 1 | Template Engine | [docs/requirements/Component-TemplateEngine.md](docs/requirements/Component-TemplateEngine.md) | Template modeling, inheritance, composition, path-qualified member addressing, override granularity, locking, alarms, flattening, semantic validation, revision hashing, and diff calculation. | +| 2 | Deployment Manager | [docs/requirements/Component-DeploymentManager.md](docs/requirements/Component-DeploymentManager.md) | Central-side deployment pipeline with deployment ID/idempotency, per-instance operation lock, state transition matrix, all-or-nothing site apply, system-wide artifact deployment with per-site status. | +| 3 | Site Runtime | [docs/requirements/Component-SiteRuntime.md](docs/requirements/Component-SiteRuntime.md) | Site-side actor hierarchy with explicit supervision strategies, staggered startup, script trust model (constrained APIs), Tell/Ask conventions, concurrency serialization, and site-wide Akka stream with per-subscriber backpressure. | +| 4 | Data Connection Layer | [docs/requirements/Component-DataConnectionLayer.md](docs/requirements/Component-DataConnectionLayer.md) | Common data connection interface (OPC UA, custom), Become/Stash connection actor model, auto-reconnect, immediate bad quality on disconnect, transparent re-subscribe, synchronous write failures, tag path resolution retry. | +| 5 | Central–Site Communication | [docs/requirements/Component-Communication.md](docs/requirements/Component-Communication.md) | Akka.NET remoting/cluster topology, 8 message patterns with per-pattern timeouts, application-level correlation IDs, transport heartbeat config, message ordering, connection failure behavior. | +| 6 | Store-and-Forward Engine | [docs/requirements/Component-StoreAndForward.md](docs/requirements/Component-StoreAndForward.md) | Buffering (transient failures only), fixed-interval retry, parking, async best-effort replication, SQLite persistence at sites. | +| 7 | External System Gateway | [docs/requirements/Component-ExternalSystemGateway.md](docs/requirements/Component-ExternalSystemGateway.md) | HTTP/REST + JSON, API key/Basic Auth, per-system timeout, dual call modes (Call/CachedCall), transient/permanent error classification, dedicated blocking I/O dispatcher, ADO.NET connection pooling. | +| 8 | Notification Service | [docs/requirements/Component-NotificationService.md](docs/requirements/Component-NotificationService.md) | SMTP with OAuth2 (M365) or Basic Auth, BCC delivery, plain text, transient/permanent SMTP error classification, store-and-forward integration. | +| 9 | Central UI | [docs/requirements/Component-CentralUI.md](docs/requirements/Component-CentralUI.md) | Blazor Server with SignalR real-time push, load balancer failover with JWT, all management workflows. | +| 10 | Security & Auth | [docs/requirements/Component-Security.md](docs/requirements/Component-Security.md) | Direct LDAP bind (LDAPS/StartTLS), JWT sessions (HMAC-SHA256, 15-min refresh, 30-min idle), role-based authorization, site-scoped permissions. | +| 11 | Health Monitoring | [docs/requirements/Component-HealthMonitoring.md](docs/requirements/Component-HealthMonitoring.md) | 30s report interval, 60s offline threshold, monotonic sequence numbers, raw error counts, tag resolution counts, dead letter monitoring. | +| 12 | Site Event Logging | [docs/requirements/Component-SiteEventLogging.md](docs/requirements/Component-SiteEventLogging.md) | SQLite storage, 30-day retention + 1GB cap, daily purge, paginated remote queries with keyword search. | +| 13 | Cluster Infrastructure | [docs/requirements/Component-ClusterInfrastructure.md](docs/requirements/Component-ClusterInfrastructure.md) | Akka.NET cluster, keep-oldest SBR with down-if-alone, min-nr-of-members=1, 2s/10s/15s failure detection, CoordinatedShutdown, automatic dual-node recovery. | +| 14 | Inbound API | [docs/requirements/Component-InboundAPI.md](docs/requirements/Component-InboundAPI.md) | POST /api/{methodName}, X-API-Key header, flat JSON, extended type system (Object/List), script-based implementations, failures-only logging. | +| 15 | Host | [docs/requirements/Component-Host.md](docs/requirements/Component-Host.md) | Single deployable binary, role-based component registration, per-component config binding (Options pattern), readiness gating, dead letter monitoring, Akka.NET bootstrap, ASP.NET Core hosting for central. | +| 16 | Commons | [docs/requirements/Component-Commons.md](docs/requirements/Component-Commons.md) | Namespace/folder convention (Types/Interfaces/Entities/Messages), shared data types, POCOs, repository interfaces, message contracts with additive-only versioning, UTC timestamp convention. | +| 17 | Configuration Database | [docs/requirements/Component-ConfigurationDatabase.md](docs/requirements/Component-ConfigurationDatabase.md) | EF Core data access, per-component repositories, unit-of-work, optimistic concurrency on deployment status, audit logging (IAuditService), migration management. | +| 18 | Management Service | [docs/requirements/Component-ManagementService.md](docs/requirements/Component-ManagementService.md) | Akka.NET ManagementActor on central, ClusterClientReceptionist registration, programmatic access to all admin operations, CLI interface. | +| 19 | CLI | [docs/requirements/Component-CLI.md](docs/requirements/Component-CLI.md) | Standalone command-line tool, System.CommandLine, HTTP transport via Management API, JSON/table output, mirrors all Management Service operations. | +| 20 | Traefik Proxy | [docs/requirements/Component-TraefikProxy.md](docs/requirements/Component-TraefikProxy.md) | Reverse proxy/load balancer fronting central cluster, active node routing via `/health/active`, automatic failover. | ### Reference Documentation diff --git a/docs/plans/2026-03-16-cluster-infrastructure-refinement-design.md b/docs/plans/2026-03-16-cluster-infrastructure-refinement-design.md index db43968..5f6dedc 100644 --- a/docs/plans/2026-03-16-cluster-infrastructure-refinement-design.md +++ b/docs/plans/2026-03-16-cluster-infrastructure-refinement-design.md @@ -1,7 +1,7 @@ # Cluster Infrastructure Refinement — Design **Date**: 2026-03-16 -**Component**: Cluster Infrastructure (`Component-ClusterInfrastructure.md`) +**Component**: Cluster Infrastructure (`docs/requirements/Component-ClusterInfrastructure.md`) **Status**: Approved ## Problem @@ -33,7 +33,7 @@ The Cluster Infrastructure doc covered topology and failover behavior but lacked | Document | Change | |----------|--------| -| `Component-ClusterInfrastructure.md` | Added 3 new sections: Split-Brain Resolution, Failure Detection Timing, Dual-Node Recovery. Updated Node Configuration to clarify both-as-seed. | +| `docs/requirements/Component-ClusterInfrastructure.md` | Added 3 new sections: Split-Brain Resolution, Failure Detection Timing, Dual-Node Recovery. Updated Node Configuration to clarify both-as-seed. | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-communication-layer-refinement-design.md b/docs/plans/2026-03-16-communication-layer-refinement-design.md index 0c869cc..2db4371 100644 --- a/docs/plans/2026-03-16-communication-layer-refinement-design.md +++ b/docs/plans/2026-03-16-communication-layer-refinement-design.md @@ -1,7 +1,7 @@ # Communication Layer Refinement — Design **Date**: 2026-03-16 -**Component**: Central–Site Communication (`Component-Communication.md`) +**Component**: Central–Site Communication (`docs/requirements/Component-Communication.md`) **Status**: Approved ## Problem @@ -36,7 +36,7 @@ The Communication Layer doc defined 8 message patterns clearly but lacked specif | Document | Change | |----------|--------| -| `Component-Communication.md` | Added 4 new sections: Message Timeouts, Transport Configuration, Message Ordering, Connection Failure Behavior | +| `docs/requirements/Component-Communication.md` | Added 4 new sections: Message Timeouts, Transport Configuration, Message Ordering, Connection Failure Behavior | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-data-connection-layer-refinement-design.md b/docs/plans/2026-03-16-data-connection-layer-refinement-design.md index 4cadd98..41182b9 100644 --- a/docs/plans/2026-03-16-data-connection-layer-refinement-design.md +++ b/docs/plans/2026-03-16-data-connection-layer-refinement-design.md @@ -1,7 +1,7 @@ # Data Connection Layer Refinement — Design **Date**: 2026-03-16 -**Component**: Data Connection Layer (`Component-DataConnectionLayer.md`) +**Component**: Data Connection Layer (`docs/requirements/Component-DataConnectionLayer.md`) **Status**: Approved ## Problem @@ -38,9 +38,9 @@ The Data Connection Layer doc covered the happy path (interface, subscriptions, | Document | Change | |----------|--------| -| `Component-DataConnectionLayer.md` | Added 4 new sections: Connection Lifecycle & Reconnection, Write Failure Handling, Tag Path Resolution, Health Reporting | -| `Component-HealthMonitoring.md` | Added tag resolution counts to monitored metrics table | -| `Component-SiteRuntime.md` | Updated SetAttribute description to note synchronous write failure errors | +| `docs/requirements/Component-DataConnectionLayer.md` | Added 4 new sections: Connection Lifecycle & Reconnection, Write Failure Handling, Tag Path Resolution, Health Reporting | +| `docs/requirements/Component-HealthMonitoring.md` | Added tag resolution counts to monitored metrics table | +| `docs/requirements/Component-SiteRuntime.md` | Updated SetAttribute description to note synchronous write failure errors | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-external-system-gateway-refinement-design.md b/docs/plans/2026-03-16-external-system-gateway-refinement-design.md index 8275e26..eccf905 100644 --- a/docs/plans/2026-03-16-external-system-gateway-refinement-design.md +++ b/docs/plans/2026-03-16-external-system-gateway-refinement-design.md @@ -1,7 +1,7 @@ # External System Gateway Refinement — Design **Date**: 2026-03-16 -**Component**: External System Gateway (`Component-ExternalSystemGateway.md`) +**Component**: External System Gateway (`docs/requirements/Component-ExternalSystemGateway.md`) **Status**: Approved ## Problem @@ -45,10 +45,10 @@ The External System Gateway doc lacked specification for the invocation protocol | Document | Change | |----------|--------| -| `Component-ExternalSystemGateway.md` | Updated External System Definition fields. Added sections: External System Call Modes (dual-mode API), Invocation Protocol, Call Timeout & Error Handling, Database Connection Management. | -| `Component-StoreAndForward.md` | Clarified that only transient failures are buffered; 4xx errors are not queued. | -| `Component-SiteRuntime.md` | Updated Script Runtime API with `ExternalSystem.Call()` and `ExternalSystem.CachedCall()`. | -| `HighLevelReqs.md` | Updated script capabilities (Section 4.4) to reflect dual call modes. | +| `docs/requirements/Component-ExternalSystemGateway.md` | Updated External System Definition fields. Added sections: External System Call Modes (dual-mode API), Invocation Protocol, Call Timeout & Error Handling, Database Connection Management. | +| `docs/requirements/Component-StoreAndForward.md` | Clarified that only transient failures are buffered; 4xx errors are not queued. | +| `docs/requirements/Component-SiteRuntime.md` | Updated Script Runtime API with `ExternalSystem.Call()` and `ExternalSystem.CachedCall()`. | +| `docs/requirements/HighLevelReqs.md` | Updated script capabilities (Section 4.4) to reflect dual call modes. | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-health-monitoring-refinement-design.md b/docs/plans/2026-03-16-health-monitoring-refinement-design.md index 23ea44b..6995d39 100644 --- a/docs/plans/2026-03-16-health-monitoring-refinement-design.md +++ b/docs/plans/2026-03-16-health-monitoring-refinement-design.md @@ -1,7 +1,7 @@ # Health Monitoring Refinement — Design **Date**: 2026-03-16 -**Component**: Health Monitoring (`Component-HealthMonitoring.md`) +**Component**: Health Monitoring (`docs/requirements/Component-HealthMonitoring.md`) **Status**: Approved ## Problem @@ -35,7 +35,7 @@ The Health Monitoring doc listed metrics and described the reporting concept but | Document | Change | |----------|--------| -| `Component-HealthMonitoring.md` | Expanded Reporting Protocol with concrete defaults and offline/online logic. Added Error Rate Metrics section. | +| `docs/requirements/Component-HealthMonitoring.md` | Expanded Reporting Protocol with concrete defaults and offline/online logic. Added Error Rate Metrics section. | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-inbound-api-refinement-design.md b/docs/plans/2026-03-16-inbound-api-refinement-design.md index c6bf25c..792a182 100644 --- a/docs/plans/2026-03-16-inbound-api-refinement-design.md +++ b/docs/plans/2026-03-16-inbound-api-refinement-design.md @@ -1,7 +1,7 @@ # Inbound API Refinement — Design **Date**: 2026-03-16 -**Component**: Inbound API (`Component-InboundAPI.md`) +**Component**: Inbound API (`docs/requirements/Component-InboundAPI.md`) **Status**: Approved ## Problem @@ -34,8 +34,8 @@ The Inbound API doc had good coverage of authentication, method definitions, and | Document | Change | |----------|--------| -| `Component-InboundAPI.md` | Added HTTP Contract section (URL structure, API key header, request/response format, extended type system). Added API Call Logging section. Updated Authentication Details to be definitive about X-API-Key header. | -| `Component-ExternalSystemGateway.md` | Updated method definition parameter/return types to note extended type system support. | +| `docs/requirements/Component-InboundAPI.md` | Added HTTP Contract section (URL structure, API key header, request/response format, extended type system). Added API Call Logging section. Updated Authentication Details to be definitive about X-API-Key header. | +| `docs/requirements/Component-ExternalSystemGateway.md` | Updated method definition parameter/return types to note extended type system support. | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-notification-service-refinement-design.md b/docs/plans/2026-03-16-notification-service-refinement-design.md index 20965a4..e858b59 100644 --- a/docs/plans/2026-03-16-notification-service-refinement-design.md +++ b/docs/plans/2026-03-16-notification-service-refinement-design.md @@ -1,7 +1,7 @@ # Notification Service Refinement — Design **Date**: 2026-03-16 -**Component**: Notification Service (`Component-NotificationService.md`) +**Component**: Notification Service (`docs/requirements/Component-NotificationService.md`) **Status**: Approved ## Problem @@ -32,7 +32,7 @@ The Notification Service doc covered notification lists and the script API but l | Document | Change | |----------|--------| -| `Component-NotificationService.md` | Expanded SMTP configuration with OAuth2 support, connection settings. Added Email Delivery Behavior section (recipient handling, error classification, rate limiting). Specified plain text content. | +| `docs/requirements/Component-NotificationService.md` | Expanded SMTP configuration with OAuth2 support, connection settings. Added Email Delivery Behavior section (recipient handling, error classification, rate limiting). Specified plain text content. | ## Alternatives Considered diff --git a/docs/plans/2026-03-16-security-auth-refinement-design.md b/docs/plans/2026-03-16-security-auth-refinement-design.md index e81a193..8edb7e5 100644 --- a/docs/plans/2026-03-16-security-auth-refinement-design.md +++ b/docs/plans/2026-03-16-security-auth-refinement-design.md @@ -1,7 +1,7 @@ # Security & Auth Refinement — Design **Date**: 2026-03-16 -**Component**: Security & Auth (`Component-Security.md`) +**Component**: Security & Auth (`docs/requirements/Component-Security.md`) **Status**: Approved ## Problem @@ -36,8 +36,8 @@ The Security & Auth doc defined roles and LDAP mapping but lacked specification | Document | Change | |----------|--------| -| `Component-Security.md` | Replaced Windows Integrated Auth with direct LDAP bind. Added Session Management, Token Lifecycle, Load Balancer Compatibility, and LDAP Connection Failure sections. | -| `HighLevelReqs.md` | Updated authentication description (Section 9.1) to reflect username/password with JWT. | +| `docs/requirements/Component-Security.md` | Replaced Windows Integrated Auth with direct LDAP bind. Added Session Management, Token Lifecycle, Load Balancer Compatibility, and LDAP Connection Failure sections. | +| `docs/requirements/HighLevelReqs.md` | Updated authentication description (Section 9.1) to reflect username/password with JWT. | ## Alternatives Considered diff --git a/docs/plans/2026-03-19-lmxfakeproxy-design.md b/docs/plans/2026-03-19-lmxfakeproxy-design.md index bb97c3e..2a7ae57 100644 --- a/docs/plans/2026-03-19-lmxfakeproxy-design.md +++ b/docs/plans/2026-03-19-lmxfakeproxy-design.md @@ -167,10 +167,10 @@ ENTRYPOINT ["dotnet", "LmxFakeProxy.dll"] ``` **Documentation updates:** -- `test_infra.md` — Add LmxFakeProxy to services table (6th service) +- `docs/test_infra/test_infra.md` — Add LmxFakeProxy to services table (6th service) - `infra/README.md` — Add to quick-start table -- New `test_infra_lmxfakeproxy.md` — Dedicated per-service doc -- `Component-DataConnectionLayer.md` — Note fake proxy availability for LmxProxy testing +- New `docs/test_infra/test_infra_lmxfakeproxy.md` — Dedicated per-service doc +- `docs/requirements/Component-DataConnectionLayer.md` — Note fake proxy availability for LmxProxy testing ## Unit Tests diff --git a/docs/plans/2026-03-19-lmxfakeproxy-implementation.md b/docs/plans/2026-03-19-lmxfakeproxy-implementation.md index 5cee062..427a8ff 100644 --- a/docs/plans/2026-03-19-lmxfakeproxy-implementation.md +++ b/docs/plans/2026-03-19-lmxfakeproxy-implementation.md @@ -1550,12 +1550,12 @@ git commit -m "feat(infra): add LmxFakeProxy Dockerfile and docker-compose servi ### Task 8: Documentation Updates **Files:** -- Modify: `test_infra.md` +- Modify: `docs/test_infra/test_infra.md` - Modify: `infra/README.md` -- Create: `test_infra_lmxfakeproxy.md` -- Modify: `Component-DataConnectionLayer.md` +- Create: `docs/test_infra/test_infra_lmxfakeproxy.md` +- Modify: `docs/requirements/Component-DataConnectionLayer.md` -**Step 1: Update test_infra.md** +**Step 1: Update docs/test_infra/test_infra.md** Add a row to the Services table: @@ -1566,7 +1566,7 @@ Add a row to the Services table: Add a bullet to the per-service documentation list: ``` -- [test_infra_lmxfakeproxy.md](test_infra_lmxfakeproxy.md) — LmxProxy fake server (OPC UA bridge) +- [test_infra_lmxfakeproxy.md](../test_infra/test_infra_lmxfakeproxy.md) — LmxProxy fake server (OPC UA bridge) ``` Update the Files section to add: @@ -1583,7 +1583,7 @@ Add a row to the quick-start table: | LmxFakeProxy (.NET gRPC) | 50051 (gRPC) | LmxProxy-compatible server bridging to OPC UA test server | ``` -**Step 3: Create test_infra_lmxfakeproxy.md** +**Step 3: Create docs/test_infra/test_infra_lmxfakeproxy.md** ```markdown # Test Infrastructure: LmxFakeProxy @@ -1664,18 +1664,18 @@ dotnet run -- --api-key my-secret-key - **Integration Tests** — End-to-end tests of the LmxProxy protocol path ``` -**Step 4: Update Component-DataConnectionLayer.md** +**Step 4: Update docs/requirements/Component-DataConnectionLayer.md** Add a note in the LmxProxy section (after the "Proto Source" paragraph, before "## Subscription Management"): ```markdown -**Test Infrastructure**: The `infra/lmxfakeproxy/` project provides a fake LmxProxy server that bridges to the OPC UA test server. It implements the full `scada.ScadaService` proto, enabling end-to-end testing of `RealLmxProxyClient` without a Windows LmxProxy deployment. See [test_infra_lmxfakeproxy.md](test_infra_lmxfakeproxy.md) for setup. +**Test Infrastructure**: The `infra/lmxfakeproxy/` project provides a fake LmxProxy server that bridges to the OPC UA test server. It implements the full `scada.ScadaService` proto, enabling end-to-end testing of `RealLmxProxyClient` without a Windows LmxProxy deployment. See [test_infra_lmxfakeproxy.md](../test_infra/test_infra_lmxfakeproxy.md) for setup. ``` **Step 5: Commit** ```bash -git add test_infra.md test_infra_lmxfakeproxy.md infra/README.md Component-DataConnectionLayer.md +git add docs/test_infra/test_infra.md docs/test_infra/test_infra_lmxfakeproxy.md infra/README.md docs/requirements/Component-DataConnectionLayer.md git commit -m "docs: add LmxFakeProxy to test infrastructure documentation" ``` diff --git a/docs/plans/generate_plans.md b/docs/plans/generate_plans.md index fc69d2b..4f2dfcb 100644 --- a/docs/plans/generate_plans.md +++ b/docs/plans/generate_plans.md @@ -13,8 +13,8 @@ This document defines the phased implementation strategy for the ScadaLink SCADA 1. **Each phase produces a testable, working increment** — no phase ends with unverifiable work. 2. **Dependencies are respected** — no component is built before its dependencies. -3. **Requirements traceability at bullet level** — every individual requirement (each bullet point, sub-bullet, and constraint) in HighLevelReqs.md must map to at least one work package. Section-level mapping is insufficient — a section like "4.4 Script Capabilities" contains ~8 distinct requirements that may land in different phases. See `docs/plans/requirements-traceability.md` for the matrix. -4. **Design decision traceability** — the Key Design Decisions in CLAUDE.md and detailed design in Component-*.md documents contain implementation constraints not present in HighLevelReqs.md (e.g., Become/Stash pattern, staggered startup, Tell vs Ask conventions, forbidden script APIs). Each must trace to a work package. +3. **Requirements traceability at bullet level** — every individual requirement (each bullet point, sub-bullet, and constraint) in docs/requirements/HighLevelReqs.md must map to at least one work package. Section-level mapping is insufficient — a section like "4.4 Script Capabilities" contains ~8 distinct requirements that may land in different phases. See `docs/plans/requirements-traceability.md` for the matrix. +4. **Design decision traceability** — the Key Design Decisions in CLAUDE.md and detailed design in docs/requirements/Component-*.md documents contain implementation constraints not present in docs/requirements/HighLevelReqs.md (e.g., Become/Stash pattern, staggered startup, Tell vs Ask conventions, forbidden script APIs). Each must trace to a work package. 5. **Split-section completeness** — when a HighLevelReqs section spans multiple phases, each phase's plan must explicitly list which bullets from that section it covers. The union across all phases must be the complete section with no gaps. 6. **Questions are tracked, not blocking** — any ambiguity discovered during plan generation is logged in `docs/plans/questions.md` and generation continues. Do not stop or wait for user input during plan generation. 7. **Codex MCP is best-effort** — if the Codex MCP tool is unavailable or errors during verification, note the skip in the plan document and continue. Do not block on external tool availability. @@ -445,8 +445,8 @@ For each phase, the implementation plan document must contain: 1. **Scope** — Components and features included 2. **Prerequisites** — Which phases/components must be complete -3. **Requirements Checklist** — A bullet-level checklist extracted from HighLevelReqs.md for every section this phase covers (see Bullet-Level Extraction below). Each bullet is a checkbox that must map to a work package. -4. **Design Constraints Checklist** — Applicable constraints from CLAUDE.md Key Design Decisions and Component-*.md documents, each mapped to a work package. +3. **Requirements Checklist** — A bullet-level checklist extracted from docs/requirements/HighLevelReqs.md for every section this phase covers (see Bullet-Level Extraction below). Each bullet is a checkbox that must map to a work package. +4. **Design Constraints Checklist** — Applicable constraints from CLAUDE.md Key Design Decisions and docs/requirements/Component-*.md documents, each mapped to a work package. 5. **Work Packages** — Numbered tasks with: - Description - Acceptance criteria (must cover every checklist bullet mapped to this work package) @@ -489,8 +489,8 @@ These are mapped to work packages and verified in acceptance criteria just like ### Generation Steps 1. Read the phase definition in this document -2. Read all referenced Component-*.md documents -3. Read referenced HighLevelReqs.md sections **line by line** — extract every bullet, sub-bullet, and constraint as a numbered requirement +2. Read all referenced docs/requirements/Component-*.md documents +3. Read referenced docs/requirements/HighLevelReqs.md sections **line by line** — extract every bullet, sub-bullet, and constraint as a numbered requirement 4. Read CLAUDE.md Key Design Decisions — extract constraints relevant to this phase's components 5. Build the Requirements Checklist and Design Constraints Checklist 6. Break sub-tasks into concrete work packages with acceptance criteria, mapping every checklist item @@ -516,8 +516,8 @@ After the orphan check passes, submit the plan to the Codex MCP tool (model: `gp **Step 1 — Requirements coverage review**: Submit the following as a single Codex prompt: - The complete phase plan document -- The full text of every HighLevelReqs.md section this phase covers -- The full text of every Component-*.md document referenced by this phase +- The full text of every docs/requirements/HighLevelReqs.md section this phase covers +- The full text of every docs/requirements/Component-*.md document referenced by this phase - The relevant Key Design Decisions from CLAUDE.md Ask Codex: *"Review this implementation plan against the provided requirements, component designs, and design constraints. Identify: (1) any requirement bullet, sub-bullet, or constraint from the source documents that is not covered by a work package or acceptance criterion in the plan, (2) any acceptance criterion that does not actually verify its linked requirement, (3) any contradictions between the plan and the source documents. List each finding with the specific source text and what is missing or wrong."* diff --git a/docs/plans/phase-0-solution-skeleton.md b/docs/plans/phase-0-solution-skeleton.md index 1d164c9..1f689d2 100644 --- a/docs/plans/phase-0-solution-skeleton.md +++ b/docs/plans/phase-0-solution-skeleton.md @@ -191,13 +191,13 @@ No business logic, actor systems, database connectivity, or web endpoints are im - `[KDD-code-5]` Per-component configuration via appsettings.json sections bound to options classes. → Directly maps to REQ-HOST-3. - `[KDD-code-6]` Options classes owned by component projects, not Commons. → Directly maps to HOST-3-4. -### From Component-Commons.md +### From docs/requirements/Component-Commons.md - `[CD-Commons-1]` Commons is referenced by all component libraries and the Host — project reference structure must reflect this. - `[CD-Commons-2]` No EF navigation property annotations on POCOs (Fluent API only in Configuration Database). - `[CD-Commons-3]` Configuration Database implements repository interfaces and maps POCOs — Phase 0 establishes the interface contract; implementation deferred. -### From Component-Host.md +### From docs/requirements/Component-Host.md - `[CD-Host-1]` Host is the composition root — references every component project to call their extension methods. - `[CD-Host-2]` Configuration Database registration (DbContext, repository wiring) is a Host responsibility — Phase 0 includes ConfigurationDatabase in Host's `AddXxx()` call chain (skeleton); full DbContext/repository wiring in Phase 1. diff --git a/docs/plans/phase-1-central-foundations.md b/docs/plans/phase-1-central-foundations.md index 272bfb8..3944004 100644 --- a/docs/plans/phase-1-central-foundations.md +++ b/docs/plans/phase-1-central-foundations.md @@ -116,7 +116,7 @@ | KDD-code-7 | Host readiness gating: /health/ready endpoint, no traffic until operational. | WP-12 | | KDD-code-8 | EF Core migrations: auto-apply in dev, manual SQL scripts for production. | WP-1 | -### From Component-ConfigurationDatabase.md +### From docs/requirements/Component-ConfigurationDatabase.md | ID | Constraint | Work Package | |----|-----------|--------------| @@ -131,7 +131,7 @@ | CD-ConfigDB-9 | Connection strings from Host's DatabaseOptions (bound from appsettings.json). | WP-1 | | CD-ConfigDB-10 | Production startup validates database schema version matches expected migration level; fail fast if not. | WP-1, WP-11 | -### From Component-Security.md +### From docs/requirements/Component-Security.md | ID | Constraint | Work Package | |----|-----------|--------------| @@ -150,7 +150,7 @@ | CD-Security-13 | Unauthorized actions return appropriate error and are not logged as audit events. | WP-9 | | CD-Security-14 | LDAP group mappings stored in configuration database, managed via Central UI (Admin role). | WP-2, WP-18 | -### From Component-CentralUI.md +### From docs/requirements/Component-CentralUI.md | ID | Constraint | Work Package | |----|-----------|--------------| @@ -160,7 +160,7 @@ | CD-CentralUI-4 | Both nodes share ASP.NET Data Protection keys (config DB or shared config). | WP-10, WP-21 | | CD-CentralUI-5 | Active debug view streams and in-progress subscriptions are lost on failover; user must re-open. | WP-21 | -### From Component-Host.md +### From docs/requirements/Component-Host.md | ID | Constraint | Work Package | |----|-----------|--------------| @@ -901,7 +901,7 @@ Most findings arose because the Codex review operated on a condensed summary tha 25. **"Section 9.3 claimed as covered but permissions not verifiable"** — Dismissed. Phase 1 defines the authorization policies (WP-9). The actual permission checks are exercised when each component's workflows are built. The policies define the permission boundaries; enforcement is cross-cutting. -26. **"WP-4 conflicts with Component-ConfigurationDatabase (instance lifecycle missing)"** — Dismissed. WP-4 AC#2 includes instance lifecycle. +26. **"WP-4 conflicts with docs/requirements/Component-ConfigurationDatabase (instance lifecycle missing)"** — Dismissed. WP-4 AC#2 includes instance lifecycle. ### Conclusion diff --git a/docs/plans/phase-2-modeling-validation.md b/docs/plans/phase-2-modeling-validation.md index 4890a25..2b60725 100644 --- a/docs/plans/phase-2-modeling-validation.md +++ b/docs/plans/phase-2-modeling-validation.md @@ -36,7 +36,7 @@ Specifically required from earlier phases: ## 3. Requirements Checklist -Each bullet extracted from HighLevelReqs.md sections covered by this phase. IDs follow the pattern `[section-N]`. +Each bullet extracted from docs/requirements/HighLevelReqs.md sections covered by this phase. IDs follow the pattern `[section-N]`. ### 3.1 Template Structure - `[3.1-1]` Machines are modeled as instances of templates. @@ -174,7 +174,7 @@ Each bullet extracted from HighLevelReqs.md sections covered by this phase. IDs ## 4. Design Constraints Checklist -Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md / Component-ConfigurationDatabase.md. +Constraints from CLAUDE.md Key Design Decisions and docs/requirements/Component-TemplateEngine.md / docs/requirements/Component-ConfigurationDatabase.md. ### From CLAUDE.md Key Design Decisions @@ -186,7 +186,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md - `[KDD-deploy-10]` Last-write-wins for concurrent template editing (no optimistic concurrency on templates). - `[KDD-deploy-12]` Naming collisions in composed feature modules are design-time errors. -### From Component-TemplateEngine.md +### From docs/requirements/Component-TemplateEngine.md - `[CD-TE-1]` Template has unique name/ID. - `[CD-TE-2]` Template cannot be deleted if referenced by instances or child templates. @@ -215,7 +215,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md - `[CD-TE-25]` Script has trigger configuration: Interval, Value Change, Conditional, or invoked by alarm/other script. - `[CD-TE-26]` Script has optional minimum time between runs. -### From Component-ConfigurationDatabase.md +### From docs/requirements/Component-ConfigurationDatabase.md - `[CD-CDB-1]` ITemplateEngineRepository covers: templates, attributes, alarms, scripts, compositions, instances, overrides, connection bindings, areas. - `[CD-CDB-2]` IDeploymentManagerRepository covers: current deployment status per instance, deployed configuration snapshots, system-wide artifact deployment status per site. @@ -488,11 +488,11 @@ Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md **Acceptance Criteria**: - Resolution order: Instance → Child Template (most derived first) → Composing Template → Composed Module (recursively). (`[3.6R-1]`, `[3.6R-2]`) -- Walk inheritance chain applying overrides at each level, respecting locks. (Component-TE flattening step 2) -- Resolve composed feature modules, applying overrides from composing templates, respecting locks. (Component-TE flattening step 3) -- Apply instance-level overrides, respecting locks. (Component-TE flattening step 4) +- Walk inheritance chain applying overrides at each level, respecting locks. (docs/requirements/Component-TE flattening step 2) +- Resolve composed feature modules, applying overrides from composing templates, respecting locks. (docs/requirements/Component-TE flattening step 3) +- Apply instance-level overrides, respecting locks. (docs/requirements/Component-TE flattening step 4) - Resolve data connection bindings — replace connection name references with concrete connection details from site. (`[3.3-8]`, `[CD-TE-11]`) -- Output a flat structure: list of attributes with resolved values and data source addresses, list of alarms with resolved trigger definitions, list of scripts with resolved code and triggers. (Component-TE flattening step 6) +- Output a flat structure: list of attributes with resolved values and data source addresses, list of alarms with resolved trigger definitions, list of scripts with resolved code and triggers. (docs/requirements/Component-TE flattening step 6) - Flattening success is a pre-deployment validation check. (`[3.11-1]`) - Test: multi-level inheritance with overrides and locks → correct resolution. - Test: nested composition with overrides → correct canonical names. @@ -547,7 +547,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md **Acceptance Criteria**: - Deployment package format is explicitly defined and versioned. -- Package contains: all resolved attributes (values, data types, data source addresses with connection details), all resolved alarms (trigger definitions with resolved attribute references), all resolved scripts (source code, trigger configuration, parameter/return definitions). (Component-TE flattening step 6) +- Package contains: all resolved attributes (values, data types, data source addresses with connection details), all resolved alarms (trigger definitions with resolved attribute references), all resolved scripts (source code, trigger configuration, parameter/return definitions). (docs/requirements/Component-TE flattening step 6) - Package includes the revision hash. (`[KDD-deploy-5]`) - Scripts are included for deployment to sites as part of flattened config. (`[4.1-3]`) - Pre-compilation validation occurs at central; actual compilation at site. (`[4.1-4]`) @@ -671,7 +671,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-TemplateEngine.md **Description**: Implement the EF Core repository for all template domain entities. **Acceptance Criteria**: -- Repository covers: templates, attributes, alarms, scripts, compositions, instances, overrides, connection bindings, areas. (`[CD-CDB-1]`) Note: shared scripts (`[CD-CDB-7]`) may be added to this repository or served by a separate interface — decide during implementation. The Component-ConfigurationDatabase.md scope for ITemplateEngineRepository does not explicitly include shared scripts; if a separate interface is warranted, create one. +- Repository covers: templates, attributes, alarms, scripts, compositions, instances, overrides, connection bindings, areas. (`[CD-CDB-1]`) Note: shared scripts (`[CD-CDB-7]`) may be added to this repository or served by a separate interface — decide during implementation. The docs/requirements/Component-ConfigurationDatabase.md scope for ITemplateEngineRepository does not explicitly include shared scripts; if a separate interface is warranted, create one. - Implementation uses DbContext internally with POCO entities from Commons. (`[CD-CDB-5]`) - Consuming components depend only on Commons interfaces. (`[CD-CDB-6]`) - Unit-of-work support: multiple operations commit in a single transaction. @@ -1096,7 +1096,7 @@ Codex MCP external verification was performed using model `gpt-5.4`. The review 2. **3.9 individual-instance deployment bullet not extracted** — Added `[3.9-6]` to Requirements Checklist. Split-section check updated. 3. **System-wide artifact deployment status in WP-24** — WP-24 is a stub; system-wide artifact deployment status is Phase 3C scope. Accepted as-is (stub level sufficient for Phase 2). 4. **Instance lifecycle concurrency missing from WP-20** — Added optimistic concurrency via rowversion to WP-20 acceptance criteria. Updated `[CD-CDB-4]` forward trace. -5. **Shared scripts repository assignment** — Clarified in WP-5 and WP-23 that shared scripts may use ITemplateEngineRepository or a separate interface; the Component-ConfigurationDatabase.md ITemplateEngineRepository scope does not explicitly include shared scripts. +5. **Shared scripts repository assignment** — Clarified in WP-5 and WP-23 that shared scripts may use ITemplateEngineRepository or a separate interface; the docs/requirements/Component-ConfigurationDatabase.md ITemplateEngineRepository scope does not explicitly include shared scripts. 6. **"No rollback" vs deployment history records** — Clarified in WP-15 and `[3.9-4]` that "no rollback" means no rollback mechanism/operation, not absence of deployment history records. Deployment records exist for audit per Configuration Database schema. 7. **Composition deletion constraint in WP-25** — Added clarifying note that composed-template deletion constraint is a logical implication of `[CD-TE-2]` (stricter but consistent interpretation). @@ -1112,6 +1112,6 @@ Codex MCP external verification was performed using model `gpt-5.4`. The review 12. **`[3.11-8]` Central UI / Design role not verified in WP-18** — Dismissed. Phase 2 provides the on-demand validation API. The Central UI integration and Design role enforcement for the validation UI are Phase 5 concerns. WP-18 correctly verifies the pipeline can be invoked without deployment; UI wiring is out of scope. 13. **`[4.5-3]` Design role gating** — Partially addressed: added Design role enforcement note to WP-5. Full UI-level role enforcement is Phase 5. 14. **`[CD-TE-9]` stream topics and UI display not verified** — Dismissed. Stream topics are Phase 3B (Akka stream); UI display is Phase 5. Phase 2 covers canonical names in triggers, scripts, and diffs which are the Phase 2 concern. -15. **Naming collision with canonical names contradicts HLR** — Dismissed. The HighLevelReqs statement "two feature modules that each define an attribute with the same name" is refined by Component-TemplateEngine.md which introduces canonical naming with module instance name prefixes. The component design is authoritative for implementation details; the HLR describes the user-facing intent (collisions are errors) while the component design specifies the mechanism (canonical names prevent false collisions). No contradiction — the component design is a refinement. +15. **Naming collision with canonical names contradicts HLR** — Dismissed. The HighLevelReqs statement "two feature modules that each define an attribute with the same name" is refined by docs/requirements/Component-TemplateEngine.md which introduces canonical naming with module instance name prefixes. The component design is authoritative for implementation details; the HLR describes the user-facing intent (collisions are errors) while the component design specifies the mechanism (canonical names prevent false collisions). No contradiction — the component design is a refinement. **Status**: Pass with corrections. All findings either addressed in the plan or dismissed with rationale. diff --git a/docs/plans/phase-3a-runtime-foundation.md b/docs/plans/phase-3a-runtime-foundation.md index 28e58f0..d615bdb 100644 --- a/docs/plans/phase-3a-runtime-foundation.md +++ b/docs/plans/phase-3a-runtime-foundation.md @@ -103,7 +103,7 @@ - [ ] `[KDD-cluster-4]` CoordinatedShutdown for graceful singleton handover. - [ ] `[KDD-cluster-5]` Automatic dual-node recovery from persistent storage. -### From Component-ClusterInfrastructure.md +### From docs/requirements/Component-ClusterInfrastructure.md - [ ] `[CD-CI-1]` Two-node cluster (active/standby) using Akka.NET Cluster. - [ ] `[CD-CI-2]` Leader election and role assignment (active vs. standby). @@ -120,7 +120,7 @@ - *Phase 3A scope*: Establish the pattern — no alarm persistence. Alarm Actors are Phase 3B, but the design must not persist alarm state. - [ ] `[CD-CI-13]` Keep-oldest SBR rationale: with two nodes, quorum-based strategies cause total shutdown. Keep-oldest with `down-if-alone` ensures at most one node runs the singleton. -### From Component-SiteRuntime.md +### From docs/requirements/Component-SiteRuntime.md - [ ] `[CD-SR-1]` Deployment Manager is an Akka.NET cluster singleton — guaranteed to run on exactly one node. - [ ] `[CD-SR-2]` Startup behavior step 1: Read all deployed configurations from local SQLite. @@ -133,7 +133,7 @@ - *Phase 3A scope*: Skeleton lifecycle — disable/enable/delete message handling in Deployment Manager. Full lifecycle with DCL/scripts is Phase 3B/3C. - [ ] `[CD-SR-9]` When Instance Actor is stopped (disable, delete, redeployment), Akka.NET automatically stops all child actors. -### From Component-Host.md +### From docs/requirements/Component-Host.md - [ ] `[CD-HOST-1]` REQ-HOST-6: Site-role Akka bootstrap with Remoting, Clustering, Persistence (SQLite), Split-Brain Resolver. - [ ] `[CD-HOST-2]` REQ-HOST-7: Site nodes use `Host.CreateDefaultBuilder` — generic `IHost`, **not** `WebApplication`. No Kestrel, no HTTP port, no web endpoints. @@ -418,7 +418,7 @@ Phase 3A is complete when **all** of the following pass: | # | Question | Context | Impact | Status | |---|----------|---------|--------|--------| -| Q-P3A-1 | What is the optimal batch size and delay for staggered Instance Actor startup? | Component-SiteRuntime.md suggests 20 with a "short delay." Actual values depend on OPC UA server capacity. | Performance tuning. Default to 20/100ms, make configurable. | Deferred — tune during Phase 3B when DCL is integrated. | +| Q-P3A-1 | What is the optimal batch size and delay for staggered Instance Actor startup? | docs/requirements/Component-SiteRuntime.md suggests 20 with a "short delay." Actual values depend on OPC UA server capacity. | Performance tuning. Default to 20/100ms, make configurable. | Deferred — tune during Phase 3B when DCL is integrated. | | Q-P3A-2 | Should the SQLite schema use a single database file or separate files per concern (configs, overrides, S&F, events)? | Single file is simpler. Separate files isolate concerns and allow independent backup/maintenance. | Schema design. | Recommend single file with separate tables. Simpler transaction management. Final decision during implementation. | | Q-P3A-3 | Should Akka.Persistence (event sourcing / snapshotting) be used for the Deployment Manager singleton, or is direct SQLite access sufficient? | Akka.Persistence adds complexity (journal, snapshots) but provides built-in recovery. Direct SQLite is simpler for this use case (singleton reads all configs on startup). | Architecture. | Recommend direct SQLite — Deployment Manager recovery is a full read-all-configs-and-rebuild pattern, not event replay. Akka.Persistence is overkill here. | diff --git a/docs/plans/phase-3b-site-io-observability.md b/docs/plans/phase-3b-site-io-observability.md index ab64ae1..e92a3bc 100644 --- a/docs/plans/phase-3b-site-io-observability.md +++ b/docs/plans/phase-3b-site-io-observability.md @@ -47,7 +47,7 @@ Phase 3B brings the site cluster to life as a fully operational data collection, ## Requirements Checklist -Each bullet extracted from HighLevelReqs.md at the individual requirement level. Checkbox items must each map to at least one work package. +Each bullet extracted from docs/requirements/HighLevelReqs.md at the individual requirement level. Checkbox items must each map to at least one work package. ### Section 2.2 — Communication: Central <-> Site @@ -416,7 +416,7 @@ Constraints from CLAUDE.md Key Design Decisions (KDD) and Component-*.md (CD) th **Description**: Implement the LmxProxy adapter wrapping the existing `LmxProxyClient` SDK behind IDataConnection. **Acceptance Criteria**: -- Implements all IDataConnection methods mapped per Component-DCL concrete type mappings. +- Implements all IDataConnection methods mapped per docs/requirements/Component-DCL concrete type mappings. - Connect: calls `ConnectAsync`, stores SessionId. - Subscribe: calls `SubscribeAsync`, processes `IAsyncEnumerable` stream, forwards updates. - Write: calls `WriteAsync`. diff --git a/docs/plans/phase-3c-deployment-store-forward.md b/docs/plans/phase-3c-deployment-store-forward.md index e066831..552f8e8 100644 --- a/docs/plans/phase-3c-deployment-store-forward.md +++ b/docs/plans/phase-3c-deployment-store-forward.md @@ -35,7 +35,7 @@ ## Requirements Checklist -Each bullet is extracted from the referenced HighLevelReqs.md sections. Items marked with a phase note indicate split-section bullets owned by another phase. +Each bullet is extracted from the referenced docs/requirements/HighLevelReqs.md sections. Items marked with a phase note indicate split-section bullets owned by another phase. ### Section 1.3 — Store-and-Forward Persistence (Site Clusters Only) @@ -134,7 +134,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-*.md documents rel - `[KDD-sf-3]` Messages not cleared on instance deletion. - `[KDD-sf-4]` CachedCall idempotency is the caller's responsibility. *(Documented in Phase 3C; enforced in Phase 7 integration.)* -### Component Design Constraints (from Component-DeploymentManager.md) +### Component Design Constraints (from docs/requirements/Component-DeploymentManager.md) - `[CD-DM-1]` Deployment flow: validate -> flatten -> send -> track. Validation failures stop the pipeline before anything is sent. - `[CD-DM-2]` Site-side idempotency on deployment ID — duplicate deployment receives "already applied" response. @@ -155,7 +155,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-*.md documents rel - `[CD-DM-17]` Enable: re-activates a disabled instance. - `[CD-DM-18]` Delete: removes running config, destroys Instance Actor and children. S&F messages not cleared. Fails if site unreachable — central does not mark deleted until site confirms. -### Component Design Constraints (from Component-StoreAndForward.md) +### Component Design Constraints (from docs/requirements/Component-StoreAndForward.md) - `[CD-SF-1]` Three message categories: external system calls, email notifications, cached database writes. - `[CD-SF-2]` Retry settings defined on the source entity (external system def, SMTP config, DB connection def), not per-message. @@ -170,7 +170,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-*.md documents rel - `[CD-SF-11]` Message format stores: message ID, category, target, payload, retry count, created at, last attempt at, status (pending/retrying/parked). - `[CD-SF-12]` Message lifecycle: attempt immediate delivery -> success removes; failure buffers -> retry loop -> success removes + notify standby; max retries exhausted -> park. -### Component Design Constraints (from Component-SiteRuntime.md — deployment-related) +### Component Design Constraints (from docs/requirements/Component-SiteRuntime.md — deployment-related) - `[CD-SR-1]` Deployment handling: receive config -> store in SQLite -> compile scripts -> create/update Instance Actor -> report result. - `[CD-SR-2]` For redeployments: existing Instance Actor and children stopped, then new Instance Actor created with updated config. Subscriptions re-established. @@ -179,7 +179,7 @@ Constraints from CLAUDE.md Key Design Decisions and Component-*.md documents rel - `[CD-SR-5]` Delete: stops Instance Actor and children, removes deployed config from SQLite. Does not clear S&F messages. - `[CD-SR-6]` Script compilation failure during deployment rejects entire deployment. No partial state applied. Failure reported to central. -### Component Design Constraints (from Component-Communication.md — deployment-related) +### Component Design Constraints (from docs/requirements/Component-Communication.md — deployment-related) - `[CD-COM-1]` Deployment pattern: request/response. No buffering at central. Unreachable site = immediate failure. - `[CD-COM-2]` Instance lifecycle pattern: request/response. Unreachable site = immediate failure. @@ -464,12 +464,12 @@ Constraints from CLAUDE.md Key Design Decisions and Component-*.md documents rel **Acceptance Criteria**: - S&F buffer depth reported as health metric (broken down by category) — integrates with Phase 3B Health Monitoring -- S&F activity logged to site event log: message queued, delivered, retried, parked (per Component-StoreAndForward.md Dependencies) +- S&F activity logged to site event log: message queued, delivered, retried, parked (per docs/requirements/Component-StoreAndForward.md Dependencies) - S&F buffer depth visible in health reports sent to central **Estimated Complexity**: S -**Requirements Traced**: `[CD-SF-1]` (categories), Component-StoreAndForward.md Dependencies (Site Event Logging, Health Monitoring) +**Requirements Traced**: `[CD-SF-1]` (categories), docs/requirements/Component-StoreAndForward.md Dependencies (Site Event Logging, Health Monitoring) --- @@ -648,7 +648,7 @@ Every work package traces to at least one requirement or design constraint: | WP-11 | `[1.3-2]`, `[1.3-4]`, `[1.3-5]`, `[KDD-sf-2]`, `[CD-SF-5]`, `[CD-SF-6]`, `[CD-SF-7]` | | WP-12 | `[5.4-1]` through `[5.4-4]`, `[KDD-sf-3]`, `[CD-SF-8]`, `[CD-SF-9]`, `[CD-SF-10]`, `[CD-COM-8]`, `[3.8.1-6]` | | WP-13 | `[3.8.1-4]`, `[3.8.1-6]`, `[KDD-sf-3]`, `[CD-SF-10]` | -| WP-14 | `[CD-SF-1]`, Component-StoreAndForward.md Dependencies | +| WP-14 | `[CD-SF-1]`, docs/requirements/Component-StoreAndForward.md Dependencies | | WP-15 | `[KDD-sf-4]`, `[CD-SF-7]` | | WP-16 | `[3.9-6]`, `[KDD-deploy-11]` | diff --git a/docs/plans/phase-4-operator-ui.md b/docs/plans/phase-4-operator-ui.md index bacd026..f1db2b3 100644 --- a/docs/plans/phase-4-operator-ui.md +++ b/docs/plans/phase-4-operator-ui.md @@ -128,7 +128,7 @@ - [ ] `[KDD-sec-4a]` Load balancer in front of central UI — UI must work behind load balancer (no sticky sessions, JWT-based). - [ ] `[KDD-deploy-10a]` Deployment status view shows current status only (no deployment history table — audit log provides history). -### From Component-CentralUI.md +### From docs/requirements/Component-CentralUI.md - [ ] `[CD-CentralUI-1]` No live machine data visualization — UI is focused on system management (except debug views, which are Phase 6). - [ ] `[CD-CentralUI-2]` Role-based access control enforced in UI: Admin, Design, Deployment with site scoping. @@ -136,7 +136,7 @@ - [ ] `[CD-CentralUI-4]` Central UI accesses configuration data via `ICentralUiRepository` (read-oriented queries). - [ ] `[CD-CentralUI-5]` Health dashboard: no historical data — current/latest status only (in-memory at central). -### From Component-HealthMonitoring.md +### From docs/requirements/Component-HealthMonitoring.md - [ ] `[CD-Health-1]` Health metrics held in memory at central — dashboard shows current/latest status only. - [ ] `[CD-Health-2]` Online recovery: site automatically marked online when health report received after offline period. @@ -145,19 +145,19 @@ - [ ] `[CD-Health-5]` No alerting — health monitoring is display-only. - [ ] `[CD-Health-6]` Error rate metrics: script errors include unhandled exceptions, timeouts, recursion limit violations. Alarm evaluation errors include all failures during condition evaluation. -### From Component-Security.md +### From docs/requirements/Component-Security.md - [ ] `[CD-Sec-1]` Admin role permissions include: manage sites, data connections, areas, LDAP mappings, API keys, system config, view audit logs. **Phase 4 covers**: sites, data connections, areas, LDAP mappings, API keys. **Phase 6 covers**: audit log viewer. System config is not a separate page — it is covered by the individual admin workflows. - [ ] `[CD-Sec-2]` Deployment role permissions include: manage instances (lifecycle), deploy, view deployment status, debug view, parked messages, event logs. Site-scoped Deployment only sees their permitted sites. **Phase 4 covers**: instance lifecycle, instance list, deployment status. **Phase 5 covers**: instance create/overrides/binding. **Phase 6 covers**: deploy action, debug view, parked messages, event logs. - [ ] `[CD-Sec-3]` Every UI action checks authenticated user's roles before proceeding. - [ ] `[CD-Sec-4]` Site-scoped Deployment checks verify target site is within user's permitted sites. -### From Component-InboundAPI.md +### From docs/requirements/Component-InboundAPI.md - [ ] `[CD-Inbound-1]` API key properties: name/label, key value, enabled/disabled flag. - [ ] `[CD-Inbound-2]` All key changes (create, enable/disable, delete) are audit logged. -### From Component-DeploymentManager.md +### From docs/requirements/Component-DeploymentManager.md - [ ] `[CD-Deploy-1]` Deployment status: pending, in-progress, success, failed — only current status stored. - [ ] `[CD-Deploy-2]` Per-instance operation lock — UI must handle "operation in progress" error gracefully. @@ -491,9 +491,9 @@ The phase is complete when all of the following pass: | # | Question | Context | Impact | Status | |---|----------|---------|--------|--------| -| Q-P4-1 | Should the API key value be auto-generated (GUID/random) or allow user-provided values? | Component-InboundAPI.md says "key value" but does not specify generation. | Phase 4, WP-5. | Open — assume auto-generated with optional copy-to-clipboard; user can regenerate. | -| Q-P4-2 | Should the health dashboard support configurable refresh intervals or always use the 30s report interval? | Component-HealthMonitoring.md specifies 30s default interval. | Phase 4, WP-9. | Open — assume display updates on every report arrival (no UI-side polling); interval is server-side config. | -| Q-P4-3 | Should area deletion cascade to child areas or require bottom-up deletion? | HighLevelReqs 3.10 says "parent-child relationships" but does not specify cascade behavior. | Phase 4, WP-3. | Open — assume cascade delete of child areas (if no instances assigned to any area in the subtree). | +| Q-P4-1 | Should the API key value be auto-generated (GUID/random) or allow user-provided values? | docs/requirements/Component-InboundAPI.md says "key value" but does not specify generation. | Phase 4, WP-5. | Open — assume auto-generated with optional copy-to-clipboard; user can regenerate. | +| Q-P4-2 | Should the health dashboard support configurable refresh intervals or always use the 30s report interval? | docs/requirements/Component-HealthMonitoring.md specifies 30s default interval. | Phase 4, WP-9. | Open — assume display updates on every report arrival (no UI-side polling); interval is server-side config. | +| Q-P4-3 | Should area deletion cascade to child areas or require bottom-up deletion? | docs/requirements/HighLevelReqs 3.10 says "parent-child relationships" but does not specify cascade behavior. | Phase 4, WP-3. | Open — assume cascade delete of child areas (if no instances assigned to any area in the subtree). | --- diff --git a/docs/plans/questions.md b/docs/plans/questions.md index 5a33136..b818c83 100644 --- a/docs/plans/questions.md +++ b/docs/plans/questions.md @@ -22,12 +22,12 @@ | Q2 | What Akka.NET version? | Latest stable 1.5.x (currently 1.5.62). | 2026-03-16 | | Q3 | Monorepo or separate repos? | Single monorepo with SLNX solution file (`.slnx`, the new XML-based format default in .NET 10). | 2026-03-16 | | Q4 | What CI/CD platform? | None for now. No CI/CD pipeline. | 2026-03-16 | -| Q5 | What LDAP server for dev/test? | GLAuth (lightweight LDAP) in Docker. See `infra/glauth/config.toml` and `test_infra_ldap.md`. | 2026-03-16 | -| Q6 | What MS SQL version and hosting? | SQL Server 2022 Developer Edition in Docker. See `infra/docker-compose.yml` and `test_infra_db.md`. | 2026-03-16 | +| Q5 | What LDAP server for dev/test? | GLAuth (lightweight LDAP) in Docker. See `infra/glauth/config.toml` and `docs/test_infra/test_infra_ldap.md`. | 2026-03-16 | +| Q6 | What MS SQL version and hosting? | SQL Server 2022 Developer Edition in Docker. See `infra/docker-compose.yml` and `docs/test_infra/test_infra_db.md`. | 2026-03-16 | | Q7 | JWT signing key storage? | `appsettings.json` (per environment). | 2026-03-16 | -| Q8 | OPC UA server for dev/test? | Azure IoT OPC PLC simulator in Docker. See `infra/opcua/nodes.json` and `test_infra_opcua.md`. | 2026-03-16 | +| Q8 | OPC UA server for dev/test? | Azure IoT OPC PLC simulator in Docker. See `infra/opcua/nodes.json` and `docs/test_infra/test_infra_opcua.md`. | 2026-03-16 | | Q10 | Target site hardware? | Windows Server 2022, 24 GB RAM, 1 TB drive, 16-core Xeon. | 2026-03-16 | -| Q9 | What is the custom protocol? Is there an existing specification or SDK? | LmxProxy — gRPC-based protocol (protobuf-net code-first, port 5050, API key auth). Client SDK: `LmxProxyClient` NuGet package. See Component-DataConnectionLayer.md for full API mapping and protocol details. | 2026-03-16 | +| Q9 | What is the custom protocol? Is there an existing specification or SDK? | LmxProxy — gRPC-based protocol (protobuf-net code-first, port 5050, API key auth). Client SDK: `LmxProxyClient` NuGet package. See docs/requirements/Component-DataConnectionLayer.md for full API mapping and protocol details. | 2026-03-16 | | Q11 | Are there specific external systems (MES, recipe manager) to integrate with for initial testing? | REST API test server (`infra/restapi/`) provides simulated external endpoints for External System Gateway and Inbound API testing. No real MES/recipe system needed for initial phases. | 2026-03-16 | | Q15 | Should the Machine Data Database schema be designed in this project, or is it out of scope? | Out of scope — Machine Data Database is a pre-existing database at customer sites. Test infra seeds sample tables/data in `infra/mssql/machinedata_seed.sql`. | 2026-03-16 | | Q13 | Who is the development team? | Solo developer with extensive Akka.NET experience and full availability. No parallelization constraints — phases are sequential. | 2026-03-16 | diff --git a/docs/plans/requirements-traceability.md b/docs/plans/requirements-traceability.md index 95941a2..930dfce 100644 --- a/docs/plans/requirements-traceability.md +++ b/docs/plans/requirements-traceability.md @@ -1,6 +1,6 @@ # Requirements Traceability Matrix -**Purpose**: Ensures every requirement from HighLevelReqs.md, every REQ-* identifier, and every design constraint from CLAUDE.md and Component-*.md maps to at least one work package in an implementation phase plan. Updated as plan documents are generated. +**Purpose**: Ensures every requirement from docs/requirements/HighLevelReqs.md, every REQ-* identifier, and every design constraint from CLAUDE.md and docs/requirements/Component-*.md maps to at least one work package in an implementation phase plan. Updated as plan documents are generated. **Traceability levels**: - **Section-level** (this document): Maps HighLevelReqs sections, REQ-* IDs, and design constraints to phases. Serves as the index. @@ -8,7 +8,7 @@ --- -## HighLevelReqs.md Sections → Phase Mapping +## docs/requirements/HighLevelReqs.md Sections → Phase Mapping | Section | Description | Phase(s) | Plan Document | Status | |---------|-------------|----------|---------------|--------| @@ -108,7 +108,7 @@ ## Design Constraints → Phase Mapping -Design decisions from CLAUDE.md Key Design Decisions and Component-*.md documents that impose implementation constraints beyond what HighLevelReqs specifies. Each is tagged `[KDD-category-N]` (Key Design Decision) or `[CD-Component-N]` (Component Design). Bullet-level extraction happens in the phase plan documents. +Design decisions from CLAUDE.md Key Design Decisions and docs/requirements/Component-*.md documents that impose implementation constraints beyond what docs/requirements/HighLevelReqs specifies. Each is tagged `[KDD-category-N]` (Key Design Decision) or `[CD-Component-N]` (Component Design). Bullet-level extraction happens in the phase plan documents. ### Architecture & Runtime diff --git a/Component-CLI.md b/docs/requirements/Component-CLI.md similarity index 100% rename from Component-CLI.md rename to docs/requirements/Component-CLI.md diff --git a/Component-CentralUI.md b/docs/requirements/Component-CentralUI.md similarity index 100% rename from Component-CentralUI.md rename to docs/requirements/Component-CentralUI.md diff --git a/Component-ClusterInfrastructure.md b/docs/requirements/Component-ClusterInfrastructure.md similarity index 100% rename from Component-ClusterInfrastructure.md rename to docs/requirements/Component-ClusterInfrastructure.md diff --git a/Component-Commons.md b/docs/requirements/Component-Commons.md similarity index 100% rename from Component-Commons.md rename to docs/requirements/Component-Commons.md diff --git a/Component-Communication.md b/docs/requirements/Component-Communication.md similarity index 100% rename from Component-Communication.md rename to docs/requirements/Component-Communication.md diff --git a/Component-ConfigurationDatabase.md b/docs/requirements/Component-ConfigurationDatabase.md similarity index 100% rename from Component-ConfigurationDatabase.md rename to docs/requirements/Component-ConfigurationDatabase.md diff --git a/Component-DataConnectionLayer.md b/docs/requirements/Component-DataConnectionLayer.md similarity index 99% rename from Component-DataConnectionLayer.md rename to docs/requirements/Component-DataConnectionLayer.md index 66562aa..9bcd98a 100644 --- a/Component-DataConnectionLayer.md +++ b/docs/requirements/Component-DataConnectionLayer.md @@ -102,7 +102,7 @@ LmxProxy is a gRPC-based protocol for communicating with LMX data servers. The D **Proto Source**: The `.proto` file originates from the LmxProxy server repository (`lmx/Proxy/Grpc/Protos/scada.proto` in ScadaBridge). The C# stubs are pre-generated and stored at `Adapters/LmxProxyGrpc/`. -**Test Infrastructure**: The `infra/lmxfakeproxy/` project provides a fake LmxProxy server that bridges to the OPC UA test server. It implements the full `scada.ScadaService` proto, enabling end-to-end testing of `RealLmxProxyClient` without a Windows LmxProxy deployment. See [test_infra_lmxfakeproxy.md](test_infra_lmxfakeproxy.md) for setup. +**Test Infrastructure**: The `infra/lmxfakeproxy/` project provides a fake LmxProxy server that bridges to the OPC UA test server. It implements the full `scada.ScadaService` proto, enabling end-to-end testing of `RealLmxProxyClient` without a Windows LmxProxy deployment. See [test_infra_lmxfakeproxy.md](../test_infra/test_infra_lmxfakeproxy.md) for setup. ## Connection Configuration Reference diff --git a/Component-DeploymentManager.md b/docs/requirements/Component-DeploymentManager.md similarity index 100% rename from Component-DeploymentManager.md rename to docs/requirements/Component-DeploymentManager.md diff --git a/Component-ExternalSystemGateway.md b/docs/requirements/Component-ExternalSystemGateway.md similarity index 100% rename from Component-ExternalSystemGateway.md rename to docs/requirements/Component-ExternalSystemGateway.md diff --git a/Component-HealthMonitoring.md b/docs/requirements/Component-HealthMonitoring.md similarity index 100% rename from Component-HealthMonitoring.md rename to docs/requirements/Component-HealthMonitoring.md diff --git a/Component-Host.md b/docs/requirements/Component-Host.md similarity index 100% rename from Component-Host.md rename to docs/requirements/Component-Host.md diff --git a/Component-InboundAPI.md b/docs/requirements/Component-InboundAPI.md similarity index 100% rename from Component-InboundAPI.md rename to docs/requirements/Component-InboundAPI.md diff --git a/Component-ManagementService.md b/docs/requirements/Component-ManagementService.md similarity index 100% rename from Component-ManagementService.md rename to docs/requirements/Component-ManagementService.md diff --git a/Component-NotificationService.md b/docs/requirements/Component-NotificationService.md similarity index 100% rename from Component-NotificationService.md rename to docs/requirements/Component-NotificationService.md diff --git a/Component-Security.md b/docs/requirements/Component-Security.md similarity index 100% rename from Component-Security.md rename to docs/requirements/Component-Security.md diff --git a/Component-SiteEventLogging.md b/docs/requirements/Component-SiteEventLogging.md similarity index 100% rename from Component-SiteEventLogging.md rename to docs/requirements/Component-SiteEventLogging.md diff --git a/Component-SiteRuntime.md b/docs/requirements/Component-SiteRuntime.md similarity index 100% rename from Component-SiteRuntime.md rename to docs/requirements/Component-SiteRuntime.md diff --git a/Component-StoreAndForward.md b/docs/requirements/Component-StoreAndForward.md similarity index 100% rename from Component-StoreAndForward.md rename to docs/requirements/Component-StoreAndForward.md diff --git a/Component-TemplateEngine.md b/docs/requirements/Component-TemplateEngine.md similarity index 100% rename from Component-TemplateEngine.md rename to docs/requirements/Component-TemplateEngine.md diff --git a/Component-TraefikProxy.md b/docs/requirements/Component-TraefikProxy.md similarity index 100% rename from Component-TraefikProxy.md rename to docs/requirements/Component-TraefikProxy.md diff --git a/HighLevelReqs.md b/docs/requirements/HighLevelReqs.md similarity index 100% rename from HighLevelReqs.md rename to docs/requirements/HighLevelReqs.md diff --git a/docs/requirements/lmxproxy_protocol.md b/docs/requirements/lmxproxy_protocol.md new file mode 100644 index 0000000..7f645ed --- /dev/null +++ b/docs/requirements/lmxproxy_protocol.md @@ -0,0 +1,360 @@ +# LmxProxy Protocol Specification + +The LmxProxy protocol is a gRPC-based SCADA read/write interface for bridging ScadaLink's Data Connection Layer to devices via an intermediary proxy server (LmxProxy). The proxy translates LmxProxy protocol operations into backend device calls (e.g., OPC UA). All communication uses HTTP/2 gRPC with Protocol Buffers. + +## Service Definition + +```protobuf +syntax = "proto3"; +package scada; + +service ScadaService { + rpc Connect(ConnectRequest) returns (ConnectResponse); + rpc Disconnect(DisconnectRequest) returns (DisconnectResponse); + rpc GetConnectionState(GetConnectionStateRequest) returns (GetConnectionStateResponse); + rpc Read(ReadRequest) returns (ReadResponse); + rpc ReadBatch(ReadBatchRequest) returns (ReadBatchResponse); + rpc Write(WriteRequest) returns (WriteResponse); + rpc WriteBatch(WriteBatchRequest) returns (WriteBatchResponse); + rpc WriteBatchAndWait(WriteBatchAndWaitRequest) returns (WriteBatchAndWaitResponse); + rpc Subscribe(SubscribeRequest) returns (stream VtqMessage); + rpc CheckApiKey(CheckApiKeyRequest) returns (CheckApiKeyResponse); +} +``` + +Proto file location: `src/ScadaLink.DataConnectionLayer/Adapters/Protos/scada.proto` + +## Connection Lifecycle + +### Session Model + +Every client must call `Connect` before performing any read, write, or subscribe operation. The server returns a session ID (32-character hex GUID) that must be included in all subsequent requests. Sessions persist until `Disconnect` is called or the server restarts — there is no idle timeout. + +### Authentication + +API key authentication is optional, controlled by server configuration: + +- **If required**: The `Connect` RPC fails with `success=false` if the API key doesn't match. +- **If not required**: All API keys are accepted (including empty). +- The API key is sent both in the `ConnectRequest.api_key` field and as an `x-api-key` gRPC metadata header on the `Connect` call. + +### Connect + +``` +ConnectRequest { + client_id: string // Client identifier (e.g., "ScadaLink-{guid}") + api_key: string // API key for authentication (empty if none) +} + +ConnectResponse { + success: bool // Whether connection succeeded + message: string // Status message + session_id: string // 32-char hex GUID (only valid if success=true) +} +``` + +The client generates `client_id` as `"ScadaLink-{Guid:N}"` for uniqueness. + +### Disconnect + +``` +DisconnectRequest { + session_id: string +} + +DisconnectResponse { + success: bool + message: string +} +``` + +Best-effort — the client calls disconnect but does not retry on failure. + +### GetConnectionState + +``` +GetConnectionStateRequest { + session_id: string +} + +GetConnectionStateResponse { + is_connected: bool + client_id: string + connected_since_utc_ticks: int64 // DateTime.UtcNow.Ticks at connect time +} +``` + +### CheckApiKey + +``` +CheckApiKeyRequest { + api_key: string +} + +CheckApiKeyResponse { + is_valid: bool + message: string +} +``` + +Standalone API key validation without creating a session. + +## Value-Timestamp-Quality (VTQ) + +The core data structure for all read and subscription results: + +``` +VtqMessage { + tag: string // Tag address + value: string // Value encoded as string (see Value Encoding) + timestamp_utc_ticks: int64 // UTC DateTime.Ticks (100ns intervals since 0001-01-01) + quality: string // "Good", "Uncertain", or "Bad" +} +``` + +### Value Encoding + +All values are transmitted as strings on the wire. Both client and server use the same parsing order: + +| Wire String | Parsed Type | Example | +|-------------|------------|---------| +| Numeric (double-parseable) | `double` | `"42.5"` → `42.5` | +| `"true"` / `"false"` (case-insensitive) | `bool` | `"True"` → `true` | +| Everything else | `string` | `"Running"` → `"Running"` | +| Empty string | `null` | `""` → `null` | + +For write operations, values are converted to strings via `.ToString()` before transmission. + +Arrays and lists are JSON-serialized (e.g., `[1,2,3]`). + +### Quality Codes + +Quality is transmitted as a case-insensitive string: + +| Wire Value | Meaning | OPC UA Status Code | +|-----------|---------|-------------------| +| `"Good"` | Value is reliable | `0x00000000` (StatusCode == 0) | +| `"Uncertain"` | Value may not be current | Non-zero, high bit clear | +| `"Bad"` | Value is unreliable or unavailable | High bit set (`0x80000000`) | + +A null or missing VTQ message is treated as Bad quality with null value and current UTC timestamp. + +### Timestamps + +- All timestamps are UTC. +- Encoded as `int64` representing `DateTime.Ticks` (100-nanosecond intervals since 0001-01-01 00:00:00 UTC). +- Client reconstructs via `new DateTime(ticks, DateTimeKind.Utc)`. + +## Read Operations + +### Read (Single Tag) + +``` +ReadRequest { + session_id: string // Valid session ID + tag: string // Tag address +} + +ReadResponse { + success: bool // Whether read succeeded + message: string // Error message if failed + vtq: VtqMessage // Value-timestamp-quality result +} +``` + +### ReadBatch (Multiple Tags) + +``` +ReadBatchRequest { + session_id: string + tags: repeated string // Tag addresses +} + +ReadBatchResponse { + success: bool // false if any tag failed + message: string // Error message + vtqs: repeated VtqMessage // Results in same order as request +} +``` + +Batch reads are **partially successful** — individual tags may have Bad quality while the overall response succeeds. If a tag read throws an exception, its VTQ is returned with Bad quality and current UTC timestamp. + +## Write Operations + +### Write (Single Tag) + +``` +WriteRequest { + session_id: string + tag: string + value: string // Value as string (parsed server-side) +} + +WriteResponse { + success: bool + message: string +} +``` + +### WriteBatch (Multiple Tags) + +``` +WriteItem { + tag: string + value: string +} + +WriteResult { + tag: string + success: bool + message: string +} + +WriteBatchRequest { + session_id: string + items: repeated WriteItem +} + +WriteBatchResponse { + success: bool // Overall success (all items must succeed) + message: string + results: repeated WriteResult // Per-item results +} +``` + +Batch writes are **all-or-nothing** at the reporting level — if any item fails, overall `success` is `false`. + +### WriteBatchAndWait (Atomic Write + Flag Polling) + +A compound operation: write values, then poll a flag tag until it matches an expected value or times out. + +``` +WriteBatchAndWaitRequest { + session_id: string + items: repeated WriteItem // Values to write + flag_tag: string // Tag to poll after writes + flag_value: string // Expected value (string comparison) + timeout_ms: int32 // Timeout in ms (default 5000 if ≤ 0) + poll_interval_ms: int32 // Poll interval in ms (default 100 if ≤ 0) +} + +WriteBatchAndWaitResponse { + success: bool // Overall operation success + message: string + write_results: repeated WriteResult // Per-item write results + flag_reached: bool // Whether flag matched before timeout + elapsed_ms: int32 // Total elapsed time +} +``` + +**Behavior:** +1. All writes execute first. If any write fails, the operation returns immediately with `success=false`. +2. If writes succeed, polls `flag_tag` at `poll_interval_ms` intervals. +3. Compares `readResult.Value?.ToString() == flag_value` (case-sensitive string comparison). +4. If flag matches before timeout: `success=true`, `flag_reached=true`. +5. If timeout expires: `success=true`, `flag_reached=false` (timeout is not an error). + +## Subscription (Server Streaming) + +### Subscribe + +``` +SubscribeRequest { + session_id: string + tags: repeated string // Tag addresses to monitor + sampling_ms: int32 // Backend sampling interval in milliseconds +} + +// Returns: stream of VtqMessage +``` + +**Behavior:** + +1. Server validates the session. Invalid session → `RpcException` with `StatusCode.Unauthenticated`. +2. Server registers monitored items on the backend (e.g., OPC UA subscriptions) for all requested tags. +3. On each value change, the server pushes a `VtqMessage` to the response stream. +4. The stream remains open indefinitely until: + - The client cancels (disposes the subscription). + - The server encounters an error (backend disconnect, etc.). + - The gRPC connection drops. +5. On stream termination, the client's `onStreamError` callback fires exactly once. + +**Client-side subscription lifecycle:** + +``` +ILmxSubscription subscription = await client.SubscribeAsync( + addresses: ["Motor.Speed", "Motor.Temperature"], + onUpdate: (tag, vtq) => { /* handle value change */ }, + onStreamError: () => { /* handle disconnect */ }); + +// Later: +await subscription.DisposeAsync(); // Cancels the stream +``` + +Disposing the subscription cancels the underlying `CancellationTokenSource`, which terminates the background stream-reading task and triggers server-side cleanup of monitored items. + +## Tag Addressing + +Tags are string addresses that identify data points. The proxy maps tag addresses to backend-specific identifiers. + +**LmxFakeProxy example** (OPC UA backend): + +Tag addresses are concatenated with a configurable prefix to form OPC UA node IDs: + +``` +Prefix: "ns=3;s=" +Tag: "Motor.Speed" +NodeId: "ns=3;s=Motor.Speed" +``` + +The prefix is configured at server startup via the `OPC_UA_PREFIX` environment variable. + +## Transport Details + +| Setting | Value | +|---------|-------| +| Protocol | gRPC over HTTP/2 | +| Default port | 50051 | +| TLS | Optional (controlled by `UseTls` connection parameter) | +| Metadata headers | `x-api-key` (sent on Connect call if API key configured) | + +### Connection Parameters + +The ScadaLink DCL configures LmxProxy connections via a string dictionary: + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `Host` | string | `"localhost"` | gRPC server hostname | +| `Port` | string (parsed as int) | `"50051"` | gRPC server port | +| `ApiKey` | string | (none) | API key for authentication | +| `SamplingIntervalMs` | string (parsed as int) | `"0"` | Backend sampling interval for subscriptions | +| `UseTls` | string (parsed as bool) | `"false"` | Use HTTPS instead of HTTP | + +## Error Handling + +| Operation | Error Mechanism | Client Behavior | +|-----------|----------------|-----------------| +| Connect | `success=false` in response | Throws `InvalidOperationException` | +| Read/ReadBatch | `success=false` in response | Throws `InvalidOperationException` | +| Write/WriteBatch | `success=false` in response | Throws `InvalidOperationException` | +| WriteBatchAndWait | `success=false` or `flag_reached=false` | Returns result (timeout is not an exception) | +| Subscribe (auth) | `RpcException` with `Unauthenticated` | Propagated to caller | +| Subscribe (stream) | Stream ends or gRPC error | `onStreamError` callback invoked; `sessionId` nullified | +| Any (disconnected) | Client checks `IsConnected` | Throws `InvalidOperationException("not connected")` | + +When a subscription stream ends unexpectedly, the client immediately nullifies its session ID, causing `IsConnected` to return `false`. The DCL adapter fires its `Disconnected` event, which triggers the reconnection cycle in the `DataConnectionActor`. + +## Implementation Files + +| Component | File | +|-----------|------| +| Proto definition | `src/ScadaLink.DataConnectionLayer/Adapters/Protos/scada.proto` | +| Client interface | `src/ScadaLink.DataConnectionLayer/Adapters/ILmxProxyClient.cs` | +| Client implementation | `src/ScadaLink.DataConnectionLayer/Adapters/RealLmxProxyClient.cs` | +| DCL adapter | `src/ScadaLink.DataConnectionLayer/Adapters/LmxProxyDataConnection.cs` | +| Client factory | `src/ScadaLink.DataConnectionLayer/Adapters/LmxProxyClientFactory.cs` | +| Server implementation | `infra/lmxfakeproxy/Services/ScadaServiceImpl.cs` | +| Session manager | `infra/lmxfakeproxy/Sessions/SessionManager.cs` | +| Tag mapper | `infra/lmxfakeproxy/TagMapper.cs` | +| OPC UA bridge interface | `infra/lmxfakeproxy/Bridge/IOpcUaBridge.cs` | +| OPC UA bridge impl | `infra/lmxfakeproxy/Bridge/OpcUaBridge.cs` | diff --git a/test_infra.md b/docs/test_infra/test_infra.md similarity index 100% rename from test_infra.md rename to docs/test_infra/test_infra.md diff --git a/test_infra_db.md b/docs/test_infra/test_infra_db.md similarity index 100% rename from test_infra_db.md rename to docs/test_infra/test_infra_db.md diff --git a/test_infra_ldap.md b/docs/test_infra/test_infra_ldap.md similarity index 100% rename from test_infra_ldap.md rename to docs/test_infra/test_infra_ldap.md diff --git a/test_infra_lmxfakeproxy.md b/docs/test_infra/test_infra_lmxfakeproxy.md similarity index 100% rename from test_infra_lmxfakeproxy.md rename to docs/test_infra/test_infra_lmxfakeproxy.md diff --git a/test_infra_opcua.md b/docs/test_infra/test_infra_opcua.md similarity index 100% rename from test_infra_opcua.md rename to docs/test_infra/test_infra_opcua.md diff --git a/test_infra_restapi.md b/docs/test_infra/test_infra_restapi.md similarity index 100% rename from test_infra_restapi.md rename to docs/test_infra/test_infra_restapi.md diff --git a/test_infra_smtp.md b/docs/test_infra/test_infra_smtp.md similarity index 100% rename from test_infra_smtp.md rename to docs/test_infra/test_infra_smtp.md diff --git a/infra/README.md b/infra/README.md index ce8dc0d..af26e22 100644 --- a/infra/README.md +++ b/infra/README.md @@ -100,11 +100,11 @@ Each tool supports `--help` for full usage. See the per-service docs below for d ## Detailed Documentation -See the project root for per-service setup guides: +See `docs/test_infra/` for per-service setup guides: -- [test_infra.md](../test_infra.md) — Master test infrastructure overview -- [test_infra_opcua.md](../test_infra_opcua.md) — OPC UA server details -- [test_infra_ldap.md](../test_infra_ldap.md) — LDAP server details -- [test_infra_db.md](../test_infra_db.md) — MS SQL database details -- [test_infra_smtp.md](../test_infra_smtp.md) — SMTP server details (Mailpit) -- [test_infra_restapi.md](../test_infra_restapi.md) — REST API server details (Flask) +- [test_infra.md](../docs/test_infra/test_infra.md) — Master test infrastructure overview +- [test_infra_opcua.md](../docs/test_infra/test_infra_opcua.md) — OPC UA server details +- [test_infra_ldap.md](../docs/test_infra/test_infra_ldap.md) — LDAP server details +- [test_infra_db.md](../docs/test_infra/test_infra_db.md) — MS SQL database details +- [test_infra_smtp.md](../docs/test_infra/test_infra_smtp.md) — SMTP server details (Mailpit) +- [test_infra_restapi.md](../docs/test_infra/test_infra_restapi.md) — REST API server details (Flask)