# Notification Service

The Notification Service is the central-only component that owns notification-list and SMTP definitions, and supplies the per-channel `INotificationDeliveryAdapter` implementations that the Notification Outbox invokes at delivery time. Sites never deliver notifications; they store-and-forward notification payloads to central, where this component's adapters perform all actual SMTP sends.

## Overview

Notification Service (#8) runs on the central cluster only. Its responsibilities split cleanly into two layers:

- **Definitions** — `NotificationList` and `SmtpConfiguration` entities stored in the central Configuration Database. Notification lists carry a `NotificationType` discriminator (`Email` now; additional types such as `Teams` are planned). Lists and SMTP config are never deployed to sites.
- **Delivery adapters** — stateless, per-type implementations of `INotificationDeliveryAdapter`. The Notification Outbox selects the adapter matching a notification's `Type`, calls `DeliverAsync`, and receives a three-way `DeliveryOutcome` (`Success` / `TransientFailure` / `PermanentFailure`). The adapter owns the full recipient-resolution, connection, authentication, send, and disconnect sequence.

The component code lives in `src/ZB.MOM.WW.ScadaBridge.NotificationService/`. The `EmailNotificationDeliveryAdapter` that consumes these primitives lives in `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/`.

## Key Concepts

### Central-only delivery

Before the current design, site nodes delivered notifications directly over SMTP. That arrangement required SMTP credentials and notification lists to be deployed to every site. The redesign inverts the path: a site script calls `Notify.To("list").Send(subject, body)`, receives a `NotificationId` GUID immediately, and the notification is store-and-forwarded to central. The Notification Outbox on central ingests it and calls the delivery adapter. Sites never open an SMTP connection.

This means:
- Credential exposure is limited to the central cluster.
- List membership is resolved at delivery time, so a list change takes effect for all future deliveries without redeploying to sites.
- The SMTP `MaxConcurrentConnections` limit is enforced at a single point.

### `NotificationType` discriminator

`NotificationList.Type` is a `NotificationType` enum value (`Email` currently). The script API `Notify.To("listName")` is type-agnostic — the calling script does not reference a type. The Notification Outbox reads the type from the central database when it picks up the notification, then selects the matching adapter by `INotificationDeliveryAdapter.Type`. Adding a new delivery channel means adding a new adapter; existing scripts continue to work.

### Per-delivery SMTP client lifetime

`MailKitSmtpClientWrapper` wraps a single `MailKit.Net.Smtp.SmtpClient`. MailKit's client is not thread-safe and holds one TCP/TLS connection. The DI registration is therefore a **factory**, not a singleton wrapper:

```csharp
services.AddSingleton<Func<ISmtpClientWrapper>>(_ => () => new MailKitSmtpClientWrapper());
```

`EmailNotificationDeliveryAdapter.SendAsync` invokes the factory at the top of each delivery attempt, runs connect → authenticate → send → disconnect on the fresh wrapper, and disposes it in a `finally` block. Each delivery pays a full TCP+TLS handshake; this is the deliberate cost of avoiding shared connection state between concurrent outbox dispatches. The factory shape allows a future pooled implementation to be slotted in without changing callers.

## Architecture

### Primitives registered by `AddNotificationService`

`ServiceCollectionExtensions.AddNotificationService` is the single DI entry point, called on the central composition root only:

```csharp
public static IServiceCollection AddNotificationService(this IServiceCollection services)
{
    services.AddOptions<NotificationOptions>()
        .BindConfiguration("ScadaBridge:Notification");

    services.AddHttpClient();
    services.AddSingleton<OAuth2TokenService>();
    services.AddSingleton<Func<ISmtpClientWrapper>>(_ => () => new MailKitSmtpClientWrapper());

    return services;
}
```

Three things are registered: the `NotificationOptions` fallback values, the `OAuth2TokenService` token cache, and the `ISmtpClientWrapper` factory. The `EmailNotificationDeliveryAdapter` itself is registered by `ZB.MOM.WW.ScadaBridge.NotificationOutbox`, which depends on this project.

### `INotificationDeliveryAdapter`

```csharp
public interface INotificationDeliveryAdapter
{
    NotificationType Type { get; }
    Task<DeliveryOutcome> DeliverAsync(
        Notification notification,
        CancellationToken cancellationToken = default);
}
```

The `DeliveryOutcome` record carries a `DeliveryResult` (`Success` / `TransientFailure` / `PermanentFailure`), `ResolvedTargets` (a snapshotted string of the concrete recipients, written to the `Notifications` audit row on success), and an `Error` string on failure.

### Email delivery sequence

`EmailNotificationDeliveryAdapter.DeliverAsync` runs this sequence, classifying every failure before returning:

1. **Resolve list** — calls `INotificationRepository.GetListByNameAsync`. An unknown list returns `Permanent` immediately (the list was deleted; retrying cannot fix it).
2. **Resolve recipients** — calls `GetRecipientsByListIdAsync`. An empty list returns `Permanent`.
3. **Resolve SMTP config** — calls `GetAllSmtpConfigurationsAsync`, takes the first row. No config returns `Permanent`.
4. **Parse TLS mode** — `SmtpTlsModeParser.Parse(smtpConfig.TlsMode)`. An unrecognised string returns `Permanent` (config fault, not a transient network condition).
5. **Validate addresses** — `EmailAddressValidator.ValidateAddresses(fromAddress, recipients)`. A malformed address returns `Permanent`.
6. **Send** — calls the private `SendAsync`, which connect/auth/send/disconnects via a fresh `ISmtpClientWrapper`.

`SendAsync` maps `SmtpCommandException` 5xx responses to `SmtpPermanentException`, then lets it propagate. `DeliverAsync` catches `SmtpPermanentException` → `Permanent`; SMTP 4xx / socket / protocol / timeout exceptions → `Transient` (via `SmtpErrorClassifier`); unclassified exceptions (e.g., OAuth2 token fetch failure) → `Permanent` (retrying a broken credential wastes token-endpoint calls).

### SMTP error classification

`SmtpErrorClassifier.Classify` uses MailKit's typed exceptions and the numeric `SmtpStatusCode` rather than message substring matching:

```csharp
public static SmtpErrorClass Classify(Exception ex, CancellationToken cancellationToken)
{
    if (ex is OperationCanceledException && cancellationToken.IsCancellationRequested)
        return SmtpErrorClass.Unknown;

    if (ex is SmtpCommandException command)
    {
        var code = (int)command.StatusCode;
        if (code >= 400 && code < 500) return SmtpErrorClass.Transient;
        if (code >= 500 && code < 600) return SmtpErrorClass.Permanent;
        return SmtpErrorClass.Unknown;
    }

    if (ex is SmtpProtocolException
        or ServiceNotConnectedException
        or SocketException
        or TimeoutException)
        return SmtpErrorClass.Transient;

    return SmtpErrorClass.Unknown;
}
```

A `Permanent` classification inside `SendAsync` is wrapped in `SmtpPermanentException` so the outer `DeliverAsync` catch filter can identify it cleanly.

### OAuth2 token lifecycle

`OAuth2TokenService.GetTokenAsync` fetches tokens for Microsoft 365 Client Credentials SMTP. Credentials are supplied as `tenantId:clientId:clientSecret`. Tokens are cached in a `ConcurrentDictionary` keyed by a SHA-256 hash of the credential string (NS-006), so distinct SMTP configurations never share a token. A per-credential `SemaphoreSlim` prevents thundering-herd refreshes. Tokens are refreshed 60 seconds before the reported `expires_in` expiry. Only the tenant is logged — the client secret and token value are never written to logs.

### Credential redaction

`CredentialRedactor.Scrub(text, credentials)` masks the full packed credential string and its trailing colon-component (password or `clientSecret`) in any text before it reaches a log line. Components shorter than 12 characters are not masked — a short username such as `root` would otherwise mask unrelated diagnostic text. All SMTP error paths in `EmailNotificationDeliveryAdapter` pass exception messages through `Scrub` before logging.

## Usage

### Script API

Site scripts do not interact with this component directly. The script surface is:

```csharp
// Returns a NotificationId immediately — does not block for delivery.
NotificationId id = Notify.To("Shift-Supervisors").Send("Tank overflow", "Tank T-03 is at 98%");

// Site-local while still in the S&F buffer; round-trips to central once forwarded.
NotificationDeliveryStatus status = Notify.Status(id);
```

`Notify.To("list")` is type-agnostic. The `NotificationId` is a GUID generated at the site. `Notify.Status` returns a `NotificationDeliveryStatus` record with `Status` (`Forwarding` site-local, or `Pending` / `Retrying` / `Delivered` / `Parked` / `Discarded` from central), `RetryCount`, `LastError`, and `DeliveredAt`.

### Registering the adapter

On the central host, both projects are registered. The Notification Outbox registers `EmailNotificationDeliveryAdapter` as a keyed or enumerated `INotificationDeliveryAdapter` and calls `AddNotificationService` to get its dependencies:

```csharp
// Central composition root (simplified)
services.AddNotificationService();
services.AddNotificationOutbox();   // registers EmailNotificationDeliveryAdapter
```

## Configuration

`NotificationOptions` is bound from `ScadaBridge:Notification`. These values are **fallbacks** — when a `SmtpConfiguration` row has a non-positive value for a field, the adapter uses the option value instead. A positive value on the row always takes precedence.

| Section | Key | Default | Description |
|---------|-----|---------|-------------|
| `ScadaBridge:Notification` | `ConnectionTimeoutSeconds` | `30` | SMTP connection/operation timeout in seconds. Applied when `SmtpConfiguration.ConnectionTimeoutSeconds` is zero or negative. |
| `ScadaBridge:Notification` | `MaxConcurrentConnections` | `5` | Maximum concurrent SMTP connections. Used as the documented default; enforcement is in `EmailNotificationDeliveryAdapter`. |

SMTP retry settings (`MaxRetries`, `RetryDelay`) live on the `SmtpConfiguration` entity and are read by the Notification Outbox dispatcher — they are not part of `NotificationOptions`.

### `SmtpConfiguration` entity fields

| Field | Type | Notes |
|-------|------|-------|
| `Host` | `string` | SMTP server hostname or IP. |
| `Port` | `int` | e.g., 587 for StartTLS, 465 for SSL. |
| `AuthType` | `string` | `basic` or `oauth2`. |
| `Credentials` | `string?` | Basic: `username:password`. OAuth2: `tenantId:clientId:clientSecret`. |
| `TlsMode` | `string?` | `None`, `StartTLS`, or `SSL`. Null/empty defaults to `StartTls`. |
| `FromAddress` | `string` | Sender address in the From header. Also the XOAUTH2 `user=` identity for M365. |
| `ConnectionTimeoutSeconds` | `int` | 0 → falls back to `NotificationOptions`. |
| `MaxConcurrentConnections` | `int` | 0 → falls back to `NotificationOptions`. |
| `MaxRetries` | `int` | Read by Notification Outbox. |
| `RetryDelay` | `TimeSpan` | Read by Notification Outbox. |

### `NotificationList` entity fields

| Field | Type | Notes |
|-------|------|-------|
| `Name` | `string` | Unique list name. Passed as `Notify.To("name")`. |
| `Type` | `NotificationType` | Enum discriminator. Currently `Email` only. |
| `Recipients` | `ICollection<NotificationRecipient>` | Resolved at delivery time by the adapter. |

Each `NotificationRecipient` carries `Name` (display) and `EmailAddress`.

## Dependencies & Interactions

- [Commons (#16)](./Commons.md) — owns `NotificationList`, `NotificationRecipient`, `SmtpConfiguration`, `Notification`, `NotificationType`, `NotificationStatus`, `INotificationRepository`, and the `NotificationSubmit` / `NotificationSubmitAck` / `NotificationStatusQuery` / `NotificationStatusResponse` / `NotificationDeliveryStatus` message contracts.
- [Configuration Database (#17)](./ConfigurationDatabase.md) — persists `NotificationList`, `NotificationRecipient`, and `SmtpConfiguration`. Implements `INotificationRepository`. The `EmailNotificationDeliveryAdapter` resolves lists and recipients via this repository at delivery time.
- [Notification Outbox (#21)](./NotificationOutbox.md) — the central dispatch counterpart. The Notification Outbox registers `EmailNotificationDeliveryAdapter`, drives retry and parking, and owns the `Notifications` audit table. Notification Service supplies the SMTP primitives (`ISmtpClientWrapper` factory, `OAuth2TokenService`, `SmtpErrorClassifier`, `CredentialRedactor`, `EmailAddressValidator`); Notification Outbox owns when and how often `DeliverAsync` is called.
- [Store-and-Forward Engine (#6)](./StoreAndForward.md) — site-side buffer. Site scripts hand notifications to the S&F engine, which forwards them to central. The Notification Service has no direct interaction with the site S&F engine; by the time `DeliverAsync` is called, the notification has already been ingested by the Notification Outbox.
- [Security & Auth (#10)](./Security.md) — Design role is required to manage notification lists and SMTP configuration.
- Design spec: [Component-NotificationService.md](../requirements/Component-NotificationService.md).

## Troubleshooting

### A notification is Parked with a permanent failure

A `PermanentFailure` outcome means `EmailNotificationDeliveryAdapter` determined that retrying cannot fix the failure. Common root causes:

| Symptom | Cause | Fix |
|---------|-------|-----|
| "Notification list '…' not found" | List was renamed or deleted after the notification was submitted. | Recreate the list or discard the notification in the Central UI Outbox page. |
| "Notification list '…' has no recipients" | List exists but has no recipient rows. | Add recipients to the list. |
| "No SMTP configuration available" | No `SmtpConfiguration` row exists. | Add an SMTP configuration in Central UI. |
| "Unknown SMTP TLS mode '…'" | `TlsMode` field contains a value other than `None`, `StartTLS`, or `SSL`. | Correct the `TlsMode` value. |
| "Invalid sender (from) email address" or "Invalid recipient email address(es)" | Malformed address in the `SmtpConfiguration.FromAddress` or in a `NotificationRecipient.EmailAddress`. | Correct the address; the adapter validates via `MailboxAddress.TryParse`. |
| SMTP 5xx reply | Server rejected the message permanently (e.g., mailbox not found, policy block). | Check the `LastError` field on the `Notifications` row. The error text has credentials redacted. |
| OAuth2 credential parse error | `Credentials` field is not in `tenantId:clientId:clientSecret` format. | Correct the credentials on the SMTP configuration. |

### Transient failures retrying indefinitely

The retry count and delay come from `SmtpConfiguration.MaxRetries` and `RetryDelay`, enforced by the Notification Outbox. Once `MaxRetries` is exhausted, the Notification Outbox moves the row to `Parked`. If a notification stays in `Retrying` longer than expected, check whether `MaxRetries` is set to a non-zero value on the `SmtpConfiguration` row and that the Notification Outbox actor is running on the active central node.

### OAuth2 token not refreshing

`OAuth2TokenService` caches tokens per credential hash. A singleton restart resets the cache; the next `GetTokenAsync` call fetches a fresh token. If token fetches fail repeatedly (network partition to `login.microsoftonline.com`, wrong tenant/client/secret), the failure surfaces as an unclassified exception in `DeliverAsync` and the notification is parked as permanent. The log line includes the tenant ID but not the secret.

## Related Documentation

- [Notification Service design specification](../requirements/Component-NotificationService.md)
- [Notification Outbox](./NotificationOutbox.md)
- [Commons](./Commons.md)
- [Configuration Database](./ConfigurationDatabase.md)
- [Store-and-Forward Engine](./StoreAndForward.md)
- [Security & Auth](./Security.md)