# LmxProxy - High Level Requirements

## 1. System Purpose

LmxProxy is a gRPC proxy service that bridges SCADA clients to AVEVA System Platform (Wonderware) via the ArchestrA MXAccess COM API. It exists because MXAccess is a 32-bit COM component that requires co-location with System Platform on a Windows machine running .NET Framework 4.8. LmxProxy isolates this constraint behind a gRPC interface, allowing modern .NET clients to access System Platform data remotely over HTTP/2.

## 2. Architecture

### 2.1 Two-Project Structure

- **ZB.MOM.WW.LmxProxy.Host** — .NET Framework 4.8, x86-only Windows service. Hosts a gRPC server (Grpc.Core) fronting the MXAccess COM API. Runs on the same machine as AVEVA System Platform.
- **ZB.MOM.WW.LmxProxy.Client** — .NET 10, AnyCPU class library. Code-first gRPC client (protobuf-net.Grpc) consumed by ScadaLink's Data Connection Layer. Packaged as a NuGet library.

### 2.2 Dual gRPC Stacks

The two projects use different gRPC implementations that are wire-compatible:

- **Host**: Proto-file-generated code via `Grpc.Core` + `Grpc.Tools`. Uses the deprecated C-core gRPC library because .NET Framework 4.8 does not support `Grpc.Net.Server`.
- **Client**: Code-first contracts via `protobuf-net.Grpc` with `[DataContract]`/`[ServiceContract]` attributes over `Grpc.Net.Client`.

Both target the same `scada.ScadaService` gRPC service definition and are wire-compatible.

### 2.3 Deployment Model

- The Host service runs on the AVEVA System Platform machine (or any machine with MXAccess access).
- Clients connect remotely over gRPC (HTTP/2) on a configurable port (default 50051).
- The Host runs as a Windows service managed by Topshelf.

## 3. Communication Protocol

### 3.1 Transport

- gRPC over HTTP/2.
- Default server port: 50051.
- Optional TLS with mutual TLS (mTLS) support.

### 3.2 RPCs

The service exposes 10 RPCs:

| RPC | Type | Description |
|-----|------|-------------|
| Connect | Unary | Establish session, returns session ID |
| Disconnect | Unary | Terminate session |
| GetConnectionState | Unary | Query MxAccess connection status |
| Read | Unary | Read single tag value |
| ReadBatch | Unary | Read multiple tag values |
| Write | Unary | Write single tag value |
| WriteBatch | Unary | Write multiple tag values |
| WriteBatchAndWait | Unary | Write values, poll flag tag until match or timeout |
| Subscribe | Server streaming | Stream tag value updates to client |
| CheckApiKey | Unary | Validate API key and return role |

### 3.3 Data Model (VTQ)

All tag values are represented as VTQ (Value, Timestamp, Quality) tuples:

- **Value**: `TypedValue` — a protobuf `oneof` carrying the value in its native type (bool, int32, int64, float, double, string, bytes, datetime, typed arrays). An unset `oneof` represents null.
- **Timestamp**: UTC `DateTime.Ticks` as `int64` (100-nanosecond intervals since 0001-01-01 00:00:00 UTC).
- **Quality**: `QualityCode` — a structured message with `uint32 status_code` (OPC UA-compatible) and `string symbolic_name`. Category derived from high bits: `0x00xxxxxx` = Good, `0x40xxxxxx` = Uncertain, `0x80xxxxxx` = Bad.

## 4. Session Lifecycle

- Clients call `Connect` with a client ID and optional API key to establish a session.
- The server returns a 32-character hex GUID as the session ID.
- All subsequent operations require the session ID for validation.
- Sessions persist until explicit `Disconnect` or server restart. There is no idle timeout.
- Session state is tracked in memory (not persisted). All sessions are lost on service restart.

## 5. Authentication & Authorization

### 5.1 API Key Authentication

- API keys are validated via the `x-api-key` gRPC metadata header.
- Keys are stored in a JSON file (`apikeys.json` by default) with hot-reload via FileSystemWatcher (1-second debounce).
- If no API key file exists, the service auto-generates a default file with two random keys (one ReadOnly, one ReadWrite).
- Authentication is enforced at the gRPC interceptor level before any service method executes.

### 5.2 Role-Based Authorization

Two roles with hierarchical permissions:

| Role | Read | Subscribe | Write |
|------|------|-----------|-------|
| ReadOnly | Yes | Yes | No |
| ReadWrite | Yes | Yes | Yes |

Write-protected methods: `Write`, `WriteBatch`, `WriteBatchAndWait`. A ReadOnly key attempting a write receives `StatusCode.PermissionDenied`.

### 5.3 TLS/Security

- TLS is optional (disabled by default in configuration, though `Tls.Enabled` defaults to `true` in the config class).
- Supports server TLS and mutual TLS (client certificate validation).
- Client CA certificate path configurable for mTLS.
- Certificate revocation checking is optional.
- Client library supports TLS 1.2 and TLS 1.3, custom CA trust stores, self-signed certificate allowance, and server name override.

## 6. Operations

### 6.1 Read

- Single tag read with configurable retry policy.
- Batch read with semaphore-controlled concurrency (default max 10 concurrent operations).
- Read timeout: 5 seconds (configurable).

### 6.2 Write

- Single tag write with retry policy. Values are sent as `TypedValue` (native protobuf types). Type mismatches between the value and the tag's expected type return a write failure.
- Batch write with semaphore-controlled concurrency.
- Write timeout: 5 seconds (configurable).
- WriteBatchAndWait: writes a batch, then polls the flag tag at a configurable interval until its value matches the expected flag value (type-aware comparison via `TypedValueEquals`) or a timeout expires. Default timeout: 5000ms, default poll interval: 100ms. Timeout is not an error — returns `flag_reached=false`.

### 6.3 Subscribe

- Server-streaming RPC. Client sends a list of tags and a sampling interval (in milliseconds).
- Server maintains a per-client bounded channel (default capacity 1000 messages).
- Updates are pushed as `VtqMessage` items on the stream.
- When the MxAccess connection drops, all subscribed clients receive a bad-quality notification.
- Subscriptions are cleaned up on client disconnect. When the last client unsubscribes from a tag, the underlying MxAccess subscription is disposed.

## 7. Connection Resilience

### 7.1 Host Auto-Reconnect

- If the MxAccess connection is lost, the Host automatically attempts reconnection at a fixed interval (default 5 seconds).
- Stored subscriptions are recreated after a successful reconnect.
- Auto-reconnect is configurable (`Connection.AutoReconnect`, default true).

### 7.2 Client Keep-Alive

- The client sends a lightweight `GetConnectionState` ping every 30 seconds.
- On keep-alive failure, the client marks the connection as disconnected and cleans up subscriptions.

### 7.3 Client Retry Policy

- Polly-based exponential backoff retry.
- Default: 3 attempts with 1-second initial delay (1s → 2s → 4s).
- Transient errors retried: Unavailable, DeadlineExceeded, ResourceExhausted, Aborted.

## 8. Health Monitoring & Metrics

### 8.1 Health Checks

Two health check implementations:

- **Basic** (`HealthCheckService`): Checks MxAccess connection state, subscription stats, and operation success rate. Returns Degraded if success rate < 50% (with > 100 operations) or client count > 100.
- **Detailed** (`DetailedHealthCheckService`): Reads a test tag (`System.Heartbeat`). Returns Unhealthy if not connected, Degraded if test tag quality is not Good or timestamp is older than 5 minutes.

### 8.2 Performance Metrics

- Per-operation tracking: Read, ReadBatch, Write, WriteBatch.
- Metrics: total count, success count, success rate, average/min/max latency, 95th percentile latency.
- Rolling buffer of 1000 latency samples per operation for percentile calculation.
- Metrics reported to logs every 60 seconds.

### 8.3 Status Web Server

- HTTP status server on port 8080 (configurable).
- Endpoints:
  - `GET /` — HTML dashboard with auto-refresh (30 seconds), color-coded status cards, operations table.
  - `GET /api/status` — JSON status report.
  - `GET /api/health` — Plain text `OK` (200) or `UNHEALTHY` (503).

### 8.4 Client Metrics

- Per-operation counts, error counts, and latency tracking (average, p95, p99).
- Rolling buffer of 1000 latency samples.
- Exposed via `ILmxProxyClient.GetMetrics()`.

## 9. Service Hosting

### 9.1 Topshelf Windows Service

- Service name: `ZB.MOM.WW.LmxProxy.Host`
- Display name: `SCADA Bridge LMX Proxy`
- Starts automatically on boot.

### 9.2 Service Recovery (Windows SCM)

| Failure | Restart Delay |
|---------|--------------|
| First | 1 minute |
| Second | 5 minutes |
| Subsequent | 10 minutes |
| Reset period | 1 day |

### 9.3 Startup Sequence

1. Load configuration from `appsettings.json` + environment variables.
2. Configure Serilog (console + file sinks).
3. Validate configuration.
4. Check/generate TLS certificates (if TLS enabled).
5. Initialize services: PerformanceMetrics, ApiKeyService, MxAccessClient, SubscriptionManager, SessionManager, HealthCheckService, StatusReportService.
6. Connect to MxAccess synchronously (timeout: 30 seconds).
7. Start auto-reconnect monitor loop (if enabled).
8. Start gRPC server on configured port.
9. Start HTTP status web server.

### 9.4 Shutdown Sequence

1. Cancel reconnect monitor (5-second wait).
2. Graceful gRPC server shutdown (10-second timeout, then kill).
3. Stop status web server (5-second wait).
4. Dispose all components in reverse order.
5. Disconnect from MxAccess (10-second timeout).

## 10. Configuration

All configuration is via `appsettings.json` bound to `LmxProxyConfiguration`. Key settings:

| Section | Setting | Default |
|---------|---------|---------|
| Root | GrpcPort | 50051 |
| Root | ApiKeyConfigFile | `apikeys.json` |
| Connection | MonitorIntervalSeconds | 5 |
| Connection | ConnectionTimeoutSeconds | 30 |
| Connection | ReadTimeoutSeconds | 5 |
| Connection | WriteTimeoutSeconds | 5 |
| Connection | MaxConcurrentOperations | 10 |
| Connection | AutoReconnect | true |
| Subscription | ChannelCapacity | 1000 |
| Subscription | ChannelFullMode | DropOldest |
| Tls | Enabled | false |
| Tls | RequireClientCertificate | false |
| WebServer | Enabled | true |
| WebServer | Port | 8080 |

Configuration is validated at startup. Invalid values cause the service to fail to start.

## 11. Logging

- Serilog with console and file sinks.
- File sink: `logs/lmxproxy-.txt`, daily rolling, 30 files retained.
- Default level: Information. Overrides: Microsoft=Warning, System=Warning, Grpc=Information.
- Enrichment: FromLogContext, WithMachineName, WithThreadId.

## 12. Constraints

- Host **must** target x86 and .NET Framework 4.8 (MXAccess is 32-bit COM).
- Host uses `Grpc.Core` (deprecated C-core library), required because .NET 4.8 does not support `Grpc.Net.Server`.
- Client targets .NET 10 and runs in ScadaLink central/site clusters.
- MxAccess COM operations require STA thread context (wrapped in `Task.Run`).
- The solution file uses `.slnx` format.

## 13. Protocol

The protocol specification is defined in `lmxproxy_updates.md`, which is the authoritative source of truth. All RPC signatures, message schemas, and behavioral specifications are per that document.

### 13.1 Value System (TypedValue)

Values are transmitted in their native protobuf types via a `TypedValue` oneof: bool, int32, int64, float, double, string, bytes, datetime (int64 UTC Ticks), and typed arrays. An unset oneof represents null. No string serialization or parsing heuristics are used.

### 13.2 Quality System (QualityCode)

Quality is a structured `QualityCode` message with `uint32 status_code` (OPC UA-compatible) and `string symbolic_name`. Supports AVEVA-aligned quality sub-codes (e.g., `BadSensorFailure` = `0x806D0000`, `GoodLocalOverride` = `0x00D80000`, `BadWaitingForInitialData` = `0x80320000`). See Component-Protocol for the full quality code table.

### 13.3 Migration from V1

The current codebase implements the v1 protocol (string-encoded values, three-state string quality). The v2 protocol is a clean break — all clients and servers will be updated simultaneously. No backward compatibility layer. This is appropriate because LmxProxy is an internal protocol with a small, controlled client count.

## 14. Component List (10 Components)

| # | Component | Description |
|---|-----------|-------------|
| 1 | GrpcServer | gRPC service implementation, session validation, request routing |
| 2 | MxAccessClient | MXAccess COM interop wrapper, connection lifecycle, read/write/subscribe |
| 3 | SessionManager | Client session tracking and lifecycle |
| 4 | Security | API key authentication, role-based authorization, TLS management |
| 5 | SubscriptionManager | Tag subscription lifecycle, channel-based update delivery, backpressure |
| 6 | Configuration | appsettings.json structure, validation, options binding |
| 7 | HealthAndMetrics | Health checks, performance metrics, status web server |
| 8 | ServiceHost | Topshelf hosting, startup/shutdown, logging setup, service recovery |
| 9 | Client | LmxProxyClient library, builder, retry, streaming, DI integration |
| 10 | Protocol | gRPC protocol specification, proto definition, code-first contracts |