ScadaBridge/docs/requirements/Component-NotificationService.md

# Component: Notification Service

## Purpose

The Notification Service is the central component that manages notification-list, SMTP, and SMS definitions and provides the per-type delivery adapters used to send notifications. It manages notification-list and delivery-channel definitions, and supplies the stateless "deliver one notification" adapter implementations that the Notification Outbox invokes at delivery time.

The Notification Service no longer delivers notifications from sites. Notification delivery has been inverted: a site script's notification is store-and-forwarded to the central cluster, and the central **Notification Outbox** owns dispatch and delivery, calling an `INotificationDeliveryAdapter` supplied by this component. See [`Component-NotificationOutbox.md`](Component-NotificationOutbox.md).

## Location

Central cluster only. The Notification Service manages definitions in the central configuration database and provides the delivery adapters that run on the central cluster. It is no longer present at site clusters, and notification definitions and SMTP configuration are no longer deployed to sites.

## Responsibilities

### Definitions (Central)
- Store notification lists in the configuration database: list name, list **type**, and type-specific targets (e.g. recipients for an `Email` list, phone numbers for an `Sms` list).
- Store email server configuration (SMTP settings).
- Store SMS provider configuration (`SmsConfiguration`: Twilio credentials and endpoint settings).
- Managed by users with the Designer role (notification lists) and Admin role (SMTP and SMS configuration).
- Notification lists, SMTP configuration, and SMS configuration are **not deployed to sites** — they exist centrally only. There is no deploy-to-sites artifact and no local SQLite copy.

### Delivery Adapters (Central)
- Provide a delivery adapter implementing `INotificationDeliveryAdapter` for each notification `Type`.
- Each adapter is a stateless "deliver one notification" implementation: it composes and sends a single notification and classifies the outcome.
- The **Email adapter** is the relocated SMTP composition and send logic — formerly run at sites, it now runs on the central cluster.
- The **SMS adapter** delivers notifications via Twilio REST to a list's phone-number recipients — see SMS Delivery Adapter below.
- Resolve a notification list name to its concrete targets (e.g. recipient addresses or phone numbers) at delivery time, on behalf of the Notification Outbox.

## Notification List Definition

Each notification list includes:
- **Name**: Unique identifier (e.g., "Maintenance-Team", "Shift-Supervisors").
- **Type**: The notification channel — `Email` or `Sms`. `Notify.To("list")` works transparently for any type — the calling script does not care about the type. The type is chosen at list creation and is **fixed** — it cannot be changed on update (prevents email/phone recipient mismatch within a list).
- **Type-specific targets**: The targets appropriate to the list type:
  - **Email list** — one or more recipient entries, each with a recipient name and email address.
  - **SMS list** — one or more recipient entries, each with a recipient name and E.164 phone number.

Lists are defined and stored centrally only. **Recipient resolution happens at central, at delivery time** — a site forwards only `(listName, subject, body)` plus provenance; the Notification Outbox asks the Notification Service to resolve the list when it dispatches the notification.

## Email Server Configuration

The SMTP configuration is defined centrally and used by the central Email delivery adapter. It is not deployed to sites. It includes:

- **Server hostname**: SMTP server address (e.g., `smtp.office365.com`).
- **Port**: SMTP port (e.g., 587 for StartTLS, 465 for SSL).
- **Authentication mode**: One of:
  - **Basic Auth**: Username and password. For on-prem SMTP relays or servers that support basic authentication.
  - **OAuth2 Client Credentials**: Tenant ID, Client ID, and Client Secret. For Microsoft 365 and other modern SMTP providers that require OAuth2. The Email adapter handles the token lifecycle internally (fetch, cache, refresh on expiry).
    - **OAuth2 authority** (optional): The token-endpoint URL requested during the client-credentials grant. When omitted it defaults to the Microsoft 365 endpoint `https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token`, with the `{tenant}` placeholder substituted from the credential's tenant id; set it explicitly to target another identity provider (e.g. a non-M365 OAuth2 SMTP provider).
    - **OAuth2 scope** (optional): The scope requested from the token endpoint. When omitted it defaults to the Microsoft 365 scope `https://outlook.office365.com/.default`; set it explicitly for another provider. Both fields are managed by Admin-role users via the CLI (`notification smtp update --oauth2-authority`/`--oauth2-scope`) and the Central UI `/notifications/smtp` page (the two inputs appear only for the OAuth2 auth type); a blank value is stored as null so the M365 default applies. Together they make "and other modern SMTP providers" true — a non-Microsoft OAuth2 relay is now configurable without a code change.
- **TLS mode**: None, StartTLS, or SSL.
- **From address**: The sender email address for all notifications (e.g., `scada-notifications@company.com`).
- **Connection timeout**: Maximum time to wait for SMTP connection (default: 30 seconds).
- **Max concurrent connections**: Maximum simultaneous SMTP connections from the central cluster (default: 5).
- **Retry settings**: Max retry count, fixed time between retries. The Notification Outbox reuses these for delivery retry of transient failures.
- **Transport**: The submission mechanism — `Smtp` (default) or `Ews`. Null/empty means `Smtp`, so every pre-existing row keeps its behavior. See EWS Transport below.

### EWS Transport (on-prem Exchange)

Email can be submitted through **Exchange Web Services** (SOAP over HTTPS) instead of SMTP, for networks where authenticated SMTP submission is not offered. The transport is chosen per configuration row via **Transport** = `Ews`; the SMTP/MailKit path is untouched and remains the default. Nothing changes for sites or scripts — delivery stays central-only and `Notify.To(...).Send(...)` behaves identically. Design rationale: [`docs/plans/2026-08-10-ews-email-transport-design.md`](../plans/2026-08-10-ews-email-transport-design.md).

**Configuration model (field reuse).** No new entity or table — the same `SmtpConfiguration` row carries both transports:

- **Host** — under `Ews`, the **full EWS endpoint URL**, an absolute `https://` URI (e.g. `https://mail.example.com/ews/exchange.asmx`), not a mail-server hostname.
- **Authentication mode** — must be `basic`. There is no OAuth2 or NTLM/Negotiate path on the EWS transport.
- **Credentials** — `username:password`, split on the **first** `:` only so a password may contain colons; the username may be domain-qualified (`domain\user`). Stored encrypted at rest via `EncryptedStringConverter`, as on the SMTP path, and projected away from every read path (`notification smtp list`, management responses, audit afterState).
- **From address**, **Connection timeout**, **Retry settings** — same meaning; the timeout becomes the per-request HTTP timeout.
- **Port**, **TLS mode**, **OAuth2 authority/scope**, **Max concurrent connections** — **unused** under `Ews`. The Central UI hides them and the CLI ignores them for this transport.

**Auth posture.** HTTP Basic over HTTPS only, sent as an explicit per-request `Authorization` header (never set on the shared, pooled `HttpClient`, which would leak the credential between callers). HTTPS is enforced in **three** places — the management write gate, the delivery adapter, and again inside the sender as defense in depth — so a Basic credential can never leave the process over cleartext even if a row is edited around a higher layer. **Revisit trigger:** if Basic is disabled on the EWS virtual directory, an auth-mode knob (Negotiate/NTLM) becomes the documented follow-on; Linux containers would then also need `gss-ntlmssp` in the image.

**Message semantics.** One `CreateItem` SOAP request per notification, `RequestServerVersion Exchange2013`, `MessageDisposition="SendOnly"`, recipients in **`BccRecipients` only**, body `BodyType="Text"`. BCC-only preserves the SMTP path's recipient handling (recipients cannot see each other — see Recipient Handling (Email) below). `SendOnly` deliberately leaves no Sent-Items copy: the central `Notifications` table is the audit record of what was sent, and a mailbox copy would grow unboundedly. All user content is XML-escaped. No SDK is used — a hand-rolled envelope over `HttpClient`, matching the no-SDK house pattern set by the SMS adapter.

**Error classification (EWS).** Classification lives inside the EWS sender, which throws typed transient/permanent exceptions that the Email adapter maps to the same three-way `DeliveryOutcome` as SMTP (there is no standalone classifier type). **What Exchange said beats what HTTP reported**: a parsed response code decides the outcome even on an HTTP 500, and only an unparseable body falls back to the status code.

- **Transient** (retry, then park after `MaxRetries`): connect/DNS/socket failures and request timeouts; HTTP 408/429/5xx on an unparseable body; and the availability-shaped response codes `ErrorServerBusy`, `ErrorInternalServerTransientError`, `ErrorTimeoutExpired`, `ErrorMailboxStoreUnavailable`, `ErrorInsufficientResources`.
- **Permanent** (park immediately): HTTP 401/403 (retrying a bad credential burns a domain account's lockout budget), 404/410/405 (wrong URL or endpoint), SOAP faults, recipient-shaped and schema response codes, and configuration defects found before the send (non-https or relative endpoint, non-Basic auth type, credentials not in `username:password` form). **Unclassified defaults to permanent**, matching the SMTP adapter's stance.
- Every surfaced message runs through `CredentialRedactor`, which masks both the packed `username:password` and its base64 Basic-auth encoding; attempts log the endpoint host and a recipient **count** only.

**Response parsing.** EWS responses are parsed with DTD processing **prohibited** and no external entity resolver, so a hostile or malfunctioning endpoint cannot drive entity expansion or external-entity retrieval from the delivery path. A malformed body is reported as unparseable rather than throwing.

**Validation layering.** The delivery adapter is the **authoritative** check — it re-validates the EWS shape at delivery time and parks the notification with a clear reason if the row is unusable, so no path can produce a silently broken send:

- **Management write gate** (`ManagementActor`, covering the CLI and the management API) rejects an unknown transport, and for `Ews` requires an absolute https `Host`, `basic` auth mode, and `username:password` credentials. This catches the common mistake of flipping an existing SMTP row — whose credentials are a bare password — to `Ews`.
- **Central UI** `/notifications/smtp` writes through the notification repository directly rather than through the management gate, so the page mirrors the same https + Basic + credential-form rules client-side before saving.
- **Transport bundle import** deliberately does **not** gate EWS shape: an imported row is applied as data, and an unusable one surfaces as a parked notification with the adapter's error. Accepted — import is an environment-migration path where the operator remaps values anyway.

## SMS Configuration

The `SmsConfiguration` entity is defined centrally and used by the central SMS delivery adapter. It mirrors `SmtpConfiguration` in structure and is not deployed to sites. It is managed by Admin-role users via the CLI (`notification sms list|update`) and the Central UI `/notifications/sms` page. It includes:

- **Account SID**: Twilio Account SID (plaintext; also appears in the API URL path).
- **Auth Token**: Twilio Auth Token, **encrypted at rest** via ASP.NET Data Protection (`EncryptedStringConverter`). The Auth Token is never returned from the list command — the listing reports it only as a `hasAuthToken` presence flag.
- **From number** (optional): Sender phone number in E.164 format (e.g., `+15551234567`). Used unless a Messaging Service SID is specified. A valid configuration carries a From number **and/or** a Messaging Service SID — at least one is required (validated in the Central UI, the CLI, and again by the delivery adapter).
- **Messaging Service SID** (optional): Twilio Messaging Service SID. When present, Twilio uses it for sender selection; used instead of the From number (so a Messaging-Service-only config needs no From number).
- **API base URL** (optional): Override for the Twilio REST API base URL (default: `https://api.twilio.com`). Allows pointing at a test/stub handler or a regional endpoint.
- **Connection timeout**: Maximum time to wait for a Twilio API response (honored per-send by the delivery adapter).
- **Max retries** / **Retry delay**: Live for SMS. The Notification Outbox dispatcher selects the retry policy by notification type — SMS notifications retry under these `SmsConfiguration` values, while Email (and every other type) reuses the central SMTP policy. When no SMS configuration row exists the dispatcher falls back to the SMTP policy, preserving the historical behavior until SMS is configured. A non-positive value is clamped to the outbox fallback (10 retries / 1-minute delay) with a Warning, matching the SMTP-policy clamp, so a misconfiguration cannot silently park SMS on the first transient failure.

The `SmsConfiguration` entity travels in Transport bundles — the Auth Token rides the encrypted `SecretsBlock` (keyed by Account SID), consistent with how SMTP credentials are bundled.

## Script API

```csharp
NotificationId id = Notify.To("listName").Send("subject", "message");
NotificationStatus status = Notify.Status(id);
```

- Available to instance scripts (via Script Execution Actors), alarm on-trigger scripts (via Alarm Execution Actors), and shared scripts (executing inline).
- `Notify.To("listName").Send(...)` is **asynchronous**: it generates a `NotificationId` (GUID) locally, hands the notification to the site Store-and-Forward Engine for forwarding to central, and returns the `NotificationId` to the script **immediately**. The script does not block waiting for delivery.
- The message body is **plain text** only. No HTML content.
- `Notify.Status(notificationId)` returns a small **status record** — the current status, retry count, last error, and key timestamps (enqueued, delivered). While the notification is still in the site Store-and-Forward buffer, the site answers the query **locally** with status `Forwarding`; once forwarded to central, the query round-trips to central and reads the `Notifications` table.
- The returned `NotificationId` is a `TrackedOperationId` — the shared Commons tracking-handle type used by all store-and-forward producers; `NotificationId` is simply the notification-domain name for it. Likewise, `Notify.Status` is a thin alias of the unified `Tracking.Status` accessor, retained for backward compatibility. This is a naming/type clarification only — notification delivery behavior is unchanged.

## Notification Delivery Behavior

Delivery is performed centrally by the Notification Outbox, which calls the `INotificationDeliveryAdapter` registered for the notification's `Type`.

### Recipient Handling (Email)
- A single email is sent per notification, with all list recipients in **BCC**. The from address is placed in the To field.
- Recipients do not see each other's email addresses.
- No per-recipient deduplication — if the same email address appears in multiple lists and a script sends to both, they receive multiple emails.

## SMS Delivery Adapter

The `SmsNotificationDeliveryAdapter` delivers notifications to an SMS list via the Twilio REST API (no Twilio SDK — uses `IHttpClientFactory` with a named `"Twilio"` `HttpClient` and HTTP Basic auth with `AccountSid:AuthToken`). It is outbound-only; true per-recipient delivery confirmation requires a status-callback webhook (out of scope for v1). "Accepted by Twilio" is treated as delivered, consistent with how the Email adapter treats "accepted by SMTP."

### Message Format
- SMS has no subject line. The message body is composed as `Subject` + newline + `Body` (whichever are present), plain text, truncated to a configurable cap (`SmsOptions.MaxMessageLength`, default 1600 — the Twilio maximum) with an ellipsis when over. Twilio segments at 160 GSM-7 characters and bills per segment.

### Per-Recipient Delivery (no BCC equivalent)
SMS has no BCC mechanism. The adapter sends one Twilio request per recipient and classifies each:

- **All accepted** → `Success`; `ResolvedTargets` is snapshotted with the accepted numbers.
- **Any transient failure** → `Transient` (the whole notification retries at the fixed interval, then Parks after max-retries). Numbers already accepted on a prior attempt are re-texted on retry — the same "re-send to all" characteristic the Email adapter already has with BCC. The recipient loop stops at the **first** transient rather than attempting the remaining recipients: every recipient accepted after the first transient would just be a guaranteed duplicate text on retry, and stopping early also bounds a black-holed endpoint to one request timeout per sweep instead of recipients × timeout. (v1 does not track per-recipient state; that is a documented future enhancement.)
- **No transient failures, mix of accepted + permanent-bad** → `Success` to the good numbers; permanently-bad numbers are recorded in `LastError`. The notification is not parked if anything got through.
- **All permanent / no recipients / no SMS config / list-not-found** → `Permanent` (Park).

### Error Classification (SMS)
A small `SmsErrorClassifier` mirrors the External System Gateway pattern:
- **Transient**: HTTP 5xx, 408, 429; `HttpRequestException`, timeout (non-caller-cancel).
- **Permanent**: other 4xx (including 401/403 bad credentials, 400 invalid/unsubscribed number).

### Error Classification (Email)
Each `Deliver(...)` call returns one of `success | transient failure | permanent failure`, consistent with the External System Gateway pattern. There is **no synchronous permanent-failure return to the script** — `Send()` returns immediately, before any delivery is attempted.

- **Transient failures** (connection refused, timeout, SMTP 4xx temporary errors): The Notification Outbox moves the row to `Retrying` and schedules another attempt per the SMTP configuration's retry settings.
- **Permanent failures** (SMTP 5xx permanent errors, e.g., mailbox not found): The Notification Outbox moves the row to `Parked` with the error in `LastError`. The notification will never deliver, and an operator can review or discard it on the Central UI Notification Outbox page.
- Retries exhausted on a transient failure also result in a `Parked` row.
- A script observes failures only by calling `Notify.Status(id)` and seeing a `Parked` status — not as a synchronous exception.
- The bullets above describe the SMTP transport. The EWS transport produces the same three outcomes from its own classification rules — see EWS Transport above.

### No Rate Limiting
- No application-level rate limiting. If the delivery endpoint enforces sending limits (e.g., Microsoft 365 throttling or Twilio rate limits), those manifest as transient failures and are retried naturally by the Notification Outbox.

## Dependencies

- **Configuration Database (MS SQL)**: Stores notification list definitions (name, type, type-specific targets), SMTP configuration, and `SmsConfiguration`.
- **Notification Outbox**: Invokes the delivery adapters supplied by this component and asks it to resolve notification lists at delivery time.
- **Security & Auth**: Designer role manages notification lists.
- **Configuration Database (via IAuditService)**: Notification list changes are audit logged.

## Interactions

- **Notification Outbox**: Consumes the per-type delivery adapters and the list-resolution service this component provides; the outbox owns dispatch, retry, parking, and status.
- **Site Runtime (Script/Alarm Execution Actors)**: Scripts invoke `Notify.To().Send()` and `Notify.Status()`. `Send()` generates a `NotificationId` and hands the notification to the site Store-and-Forward Engine; it does not contact this component synchronously.
- **Store-and-Forward Engine (site)**: Forwards a script's notification to central; the central Notification Outbox ingests it for delivery. The Notification Service does not interact with the site Store-and-Forward Engine directly.