From 00d8a923af891710e5b08a0a143a9fda5d11033f Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 10 Aug 2026 06:56:56 -0400 Subject: [PATCH] docs(notifications): EWS transport docs; close Q12 as superseded; design-doc corrections from execution reviews --- CLAUDE.md | 1 + README.md | 2 +- docs/deployment/production-checklist.md | 5 +- .../2026-08-10-ews-email-transport-design.md | 61 +++++++++++++------ ...26-08-10-ews-email-transport.md.tasks.json | 20 +++--- docs/plans/phase-7-integrations.md | 2 +- docs/plans/questions.md | 2 +- .../Component-NotificationService.md | 32 ++++++++++ docs/test_infra/test_infra_smtp.md | 2 + src/ZB.MOM.WW.ScadaBridge.CLI/README.md | 4 +- .../Import/BundleImporter.cs | 3 + 11 files changed, 102 insertions(+), 32 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8c40ddc4..d43c2789 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -106,6 +106,7 @@ spec for each is `docs/requirements/Component-.md`, and `README.md` carrie - Dual call modes: `ExternalSystem.Call()` (synchronous) and `ExternalSystem.CachedCall()` (store-and-forward on transient failure). - Error classification: HTTP 5xx/408/429/connection errors = transient; other 4xx = permanent (returned to script). - Notification Service: SMTP with OAuth2 Client Credentials (Microsoft 365) or Basic Auth. BCC delivery, plain text. +- Email delivery has **two transports**, selected per config row by `SmtpConfiguration.Transport` (null/`Smtp` = default): SMTP (MailKit, Basic/OAuth2) or on-prem Exchange **EWS** (no-SDK `CreateItem` SOAP over `HttpClient`, Basic-over-HTTPS only — https enforced at write gate, adapter AND sender — BCC-only recipients, `SendOnly` so no Sent-Items copy). Under `Ews`, `Host` is the full EWS URL, `Credentials` is `username:password`, and Port/TlsMode/OAuth2*/MaxConcurrentConnections are unused. Site-facing behavior is unchanged (delivery stays central-only). Q12 (M365 OAuth2 tenant) is closed as **superseded** — the OAuth2 SMTP path stays config-selectable but untested against a live tenant. Design: `docs/plans/2026-08-10-ews-email-transport-design.md`. - Notification delivery is central-only: sites store-and-forward notifications to the central cluster (target = central, not SMTP); sites never talk to SMTP. Notification lists and SMTP config are no longer deployed to sites; recipient resolution happens at central, at delivery time. - Notification lists carry a `Type` discriminator (`Email` and `Sms`). `Notify.To("list")` is type-agnostic; delivery is via per-type `INotificationDeliveryAdapter` (Email via SMTP; Sms via Twilio REST — `SmsNotificationDeliveryAdapter`, no SDK, one POST per recipient, per-recipient rollup). List Type is fixed after creation. - `Notify.Send` is async and **enqueue-only** — it buffers the notification into the local SQLite S&F store and returns a `NotificationId` (GUID, idempotency key) status handle immediately; it never runs the forwarder's central Ask inline on the script thread (`deferToSweep: true` buffers due-immediately + kicks a background sweep), so its worst-case latency is the local insert whether central is up or down. It enqueues with `maxRetries: 0` (the "no limit" escape hatch), so notifications retry until central acks and are **never parked for retry exhaustion** — only a corrupt payload parks them (arch-review 02, Tasks 13/14). `Notify.Status(notificationId)` returns a status record (status, retry count, last error, key timestamps); answered site-locally as `Forwarding` while still in the site S&F buffer, otherwise round-trips to central. diff --git a/README.md b/README.md index 31ce7414..64e9fdba 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ Both stacks share the infrastructure services in [`infra/`](infra/) (MS SQL, LDA | 5 | Central–Site Communication | [docs/requirements/Component-Communication.md](docs/requirements/Component-Communication.md) | Dual transport: Akka.NET ClusterClient (command/control) + gRPC server-streaming (real-time data). 9 message patterns with per-pattern timeouts, SiteStreamGrpcServer/Client, application-level correlation IDs, transport heartbeat config, gRPC keepalive, message ordering, connection failure behavior. The gRPC stream additively carries the read-only native alarm mirror (computed + native OPC UA / MxAccess) via the enriched `AlarmStateUpdate`. | | 6 | Store-and-Forward Engine | [docs/requirements/Component-StoreAndForward.md](docs/requirements/Component-StoreAndForward.md) | Buffering (transient failures only), fixed-interval retry, parking, async best-effort replication, SQLite persistence at sites. | | 7 | External System Gateway | [docs/requirements/Component-ExternalSystemGateway.md](docs/requirements/Component-ExternalSystemGateway.md) | HTTP/REST + JSON, API key/Basic Auth, per-system timeout, dual call modes (Call/CachedCall), transient/permanent error classification, dedicated blocking I/O dispatcher, ADO.NET connection pooling. | -| 8 | Notification Service | [docs/requirements/Component-NotificationService.md](docs/requirements/Component-NotificationService.md) | Central-only — manages typed notification-list, SMTP, and SMS definitions; supplies per-type delivery adapters (Email via SMTP with OAuth2 (M365) or Basic Auth, BCC, plain text; SMS via Twilio REST, per-recipient, outbound-only); delivery performed by the Notification Outbox. | +| 8 | Notification Service | [docs/requirements/Component-NotificationService.md](docs/requirements/Component-NotificationService.md) | Central-only — manages typed notification-list, SMTP, and SMS definitions; supplies per-type delivery adapters (Email via SMTP with OAuth2 (M365) or Basic Auth, or on-prem Exchange EWS, BCC, plain text; SMS via Twilio REST, per-recipient, outbound-only); delivery performed by the Notification Outbox. | | 9 | Central UI | [docs/requirements/Component-CentralUI.md](docs/requirements/Component-CentralUI.md) | Blazor Server with SignalR real-time push, load balancer failover with JWT, all management workflows. Custom-content modal host (`DialogService.ShowAsync`) with focus-trap/restore; dark-mode CSS-variable token layer (`[data-bs-theme="dark"]` overriding `ZB.MOM.WW.Theme` tokens in `site.css`, `localStorage`-persisted, SSR no-flash); reusable presentational components `OffsetPager`, `KeysetPager`, and `DateTimeRangeFilter` adopted across report/audit pages. | | 10 | Security & Auth | [docs/requirements/Component-Security.md](docs/requirements/Component-Security.md) | Direct LDAP bind (LDAPS/StartTLS), JWT sessions (HMAC-SHA256, 15-min refresh, 30-min idle), role-based authorization (incl. the `Operator`/`Verifier` two-person secured-write roles + policies), site-scoped permissions. | | 11 | Health Monitoring | [docs/requirements/Component-HealthMonitoring.md](docs/requirements/Component-HealthMonitoring.md) | 30s report interval, 60s offline threshold, monotonic sequence numbers, raw error counts, tag resolution counts, dead letter monitoring. | diff --git a/docs/deployment/production-checklist.md b/docs/deployment/production-checklist.md index b5ef6c62..9248ee61 100644 --- a/docs/deployment/production-checklist.md +++ b/docs/deployment/production-checklist.md @@ -44,6 +44,8 @@ - [ ] Windows Service account has minimum necessary permissions - [ ] Log directory permissions restrict access to service account and administrators - [ ] SMTP credentials use OAuth2 Client Credentials (preferred) or secure Basic Auth +- [ ] EWS transport (`Transport=Ews`): endpoint is an absolute `https://` URL and auth mode is Basic — Basic requires TLS and this is enforced at the write gate, the delivery adapter, and the sender +- [ ] EWS transport: the Exchange service-account password is rotated on the account-owner's schedule, and the SMTP configuration row is updated in the same change - [ ] API keys for Inbound API are generated with sufficient entropy (32+ chars) ### Network @@ -51,7 +53,8 @@ - [ ] DNS resolution works between all cluster nodes - [ ] Firewall rules permit Akka.NET remoting (TCP 8081) - [ ] Firewall rules permit LDAP (TCP 636 for LDAPS) -- [ ] Firewall rules permit SMTP (TCP 587 for TLS) +- [ ] Firewall rules permit SMTP (TCP 587 for TLS) — SMTP transport only +- [ ] EWS transport (`Transport=Ews`): firewall rules permit outbound HTTPS (TCP 443) from central nodes to the Exchange CAS instead of SMTP 587 - [ ] Firewall rules permit SQL Server (TCP 1433) from central nodes only - [ ] Load balancer health check configured against `/health/ready` diff --git a/docs/plans/2026-08-10-ews-email-transport-design.md b/docs/plans/2026-08-10-ews-email-transport-design.md index ee1f525c..4197c26e 100644 --- a/docs/plans/2026-08-10-ews-email-transport-design.md +++ b/docs/plans/2026-08-10-ews-email-transport-design.md @@ -1,6 +1,6 @@ # EWS Email Transport for the Notification Outbox — Design -**Date:** 2026-08-10 · **Status:** Designed, not implemented · **Owner decisions captured below.** +**Date:** 2026-08-10 · **Status:** Implemented on `feature/ews-email-transport`; manual live gate pending · **Owner decisions captured below.** ## 1. Context — the pending task this supersedes @@ -71,10 +71,11 @@ call (~200 lines including classification). - `Host` — the **full EWS endpoint URL** (absolute `https://` URI, validated). - `AuthType` — must be `basic`; `Credentials` stays `username:password` where username may be `domain\user` (split on the **first** `:` only, as today). Same storage treatment as SMTP - credentials (persisted server-side, projected away from all read paths by - `ManagementActor`'s credential-free projection; `CredentialRedactor.Scrub` on every error - message). The pre-existing asymmetry with `SmsConfiguration`'s Data-Protection-encrypted - `AuthToken` is noted but out of scope here. + credentials: `SmtpConfiguration.Credentials` is **encrypted at rest** via + `EncryptedStringConverter` (wired in `ScadaBridgeDbContext.ApplySecretColumnEncryption`, + `ConfigurationDatabase`), exactly like `SmsConfiguration.AuthToken` — there is no asymmetry + between them. It is additionally projected away from all read paths by `ManagementActor`'s + credential-free projection, and `CredentialRedactor.Scrub` runs on every error message. - `FromAddress`, `MaxRetries`, `RetryDelay`, `ConnectionTimeoutSeconds` — same meaning (timeout becomes the `HttpClient` request timeout). - `Port`, `TlsMode`, `OAuth2Authority`, `OAuth2Scope`, `MaxConcurrentConnections` — unused for @@ -103,15 +104,21 @@ call (~200 lines including classification). ### 4.3 Error classification (mirrors `SmtpErrorClassifier` / ESG conventions) - **Transient** (retry → park after `MaxRetries`): connect/DNS/socket/timeout failures; HTTP - 5xx/408/429; SOAP fault codes that are load/availability-shaped (`ErrorServerBusy`, - `ErrorInternalServerTransientError`, `ErrorTimeoutExpired`, `ErrorMailboxStoreUnavailable`). + 5xx/408/429; response/fault codes that are load/availability-shaped (`ErrorServerBusy`, + `ErrorInternalServerTransientError`, `ErrorTimeoutExpired`, `ErrorMailboxStoreUnavailable`, + `ErrorInsufficientResources`). - **Permanent** (park immediately): HTTP 401/403 (credential/authorization — retrying burns - lockout budget on a domain account), 404 (wrong URL), SOAP schema faults, recipient-shaped - response codes (`ErrorInvalidRecipients`, `ErrorMessageSizeExceeded`), malformed-config - findings (bad URL, bad credential form) — same "unclassified defaults to permanent" stance the - SMTP adapter takes. -- New pure `EwsErrorClassifier` alongside `SmtpErrorClassifier`; every surfaced message runs - through `CredentialRedactor.Scrub`. + lockout budget on a domain account), 404/410/405 (wrong URL or endpoint), SOAP schema faults, + recipient-shaped response codes (`ErrorInvalidRecipients`, `ErrorMessageSizeExceeded`), + malformed-config findings (bad URL, bad credential form) — same "unclassified defaults to + permanent" stance the SMTP adapter takes. +- **As built:** there is no standalone `EwsErrorClassifier` type. Classification lives inside + `EwsSoapMailSender`, which throws typed `EwsTransientException` / `EwsPermanentException` that + the adapter maps to `DeliveryOutcome`; the pure `EwsResponseParser` stays judgement-free, + reporting only the response shape and code (and parsing with DTD processing prohibited). A + parsed response code beats the HTTP status — only an unparseable body falls back to the status + code. Every surfaced message runs through `CredentialRedactor.Scrub`, plus a mask of the + base64 Basic-auth value. ### 4.4 Management surfaces - **CLI:** `notification smtp update` gains `--transport smtp|ews` (existing `--host`, @@ -123,12 +130,17 @@ call (~200 lines including classification). - **Transport bundles:** `SmtpConfiguration` already travels in the encrypted SecretsBlock; the new field rides along additively (`schemaVersion` additive rules). The existing round-trip guard (arch-review T8 style) is extended to cover `Transport` so export/import can't silently - drop it. + drop it. **Discovered during implementation:** `OAuth2Authority` and `OAuth2Scope` (shipped + 2026-07) were **already** being silently dropped by bundle export/import — `SmtpConfigDto` + never carried them. Fixed in the same slice as `Transport` (DTO, serializer, + `BundleImporter.ApplySmtpFields`, `ArtifactDiff`), and the round-trip guard reseeded so all + three fields are now pinned. ### 4.5 Testing (owner decision: fake stub + live gate) - **Unit:** `EwsSoapMailSender` against an in-process fake EWS endpoint (Kestrel `TestServer`) asserting the Basic header, envelope shape (BCC-only, SendOnly, escaping) and driving canned - `CreateItemResponse` success / SOAP-fault / HTTP-error bodies through `EwsErrorClassifier`; + `CreateItemResponse` success / SOAP-fault / HTTP-error bodies through the sender's + classification (see §4.3 — no standalone classifier type); adapter-level tests for the transport branch and outcome mapping; validator tests for the new config rules. All macOS-runnable — no NTLM, no network. - **Live gate (manual, one-off):** from the dev Mac, configure `Transport=Ews` with the @@ -153,7 +165,22 @@ call (~200 lines including classification). - Removing the SMTP/OAuth2 path (stays config-selectable; Approach "additional transport" was the owner's choice). - NTLM/Negotiate auth mode (documented follow-on trigger: Basic disabled on the EWS vdir). -- Encrypting `SmtpConfiguration.Credentials` at rest (pre-existing posture, tracked separately - if desired). - HTML bodies, attachments, per-recipient sends, Sent-Items copies. - Site-side anything — notification delivery remains central-only. + +## 6. Follow-ups (from execution reviews) + +Raised while implementing; none blocking, none scheduled here. + +1. **Catch-chain duplication in `EmailNotificationDeliveryAdapter`.** The SMTP and EWS branches + each carry a near-identical permanent / caller-cancel / transient / unclassified catch chain + differing only in exception types and log wording. Extract a shared helper **before** a third + transport is added. +2. **Bundle-import shape validation parity.** Import applies `SmtpConfigDto` fields as data and + does not run the EWS shape gate that `ManagementActor` (CLI/API) and the Central UI apply. The + delivery adapter is authoritative and parks an unusable row with a clear reason, so this is + accepted for now rather than duplicating the gate a fourth time. +3. **`CredentialRedactor.MinSecretLength` = 12.** A standalone password shorter than that is not + scrubbed on its own; EWS messages are still covered by the packed `username:password` and + base64 Basic-auth scrubs, which comfortably exceed the floor. Revisit if a code path ever + surfaces a bare short password. diff --git a/docs/plans/2026-08-10-ews-email-transport.md.tasks.json b/docs/plans/2026-08-10-ews-email-transport.md.tasks.json index dd29eb3e..d6b01d50 100644 --- a/docs/plans/2026-08-10-ews-email-transport.md.tasks.json +++ b/docs/plans/2026-08-10-ews-email-transport.md.tasks.json @@ -1,16 +1,16 @@ { "planPath": "docs/plans/2026-08-10-ews-email-transport.md", "tasks": [ - {"id": 1, "subject": "Task 1: EmailTransport enum + parser", "status": "pending"}, - {"id": 2, "subject": "Task 2: SmtpConfiguration.Transport entity + EF mapping + migration", "status": "pending"}, - {"id": 3, "subject": "Task 3: EWS SOAP envelope builder + response parser", "status": "pending"}, - {"id": 4, "subject": "Task 4: EwsSoapMailSender + DI registration", "status": "pending", "blockedBy": [3]}, - {"id": 5, "subject": "Task 5: EmailNotificationDeliveryAdapter transport branch", "status": "pending", "blockedBy": [1, 2, 4]}, - {"id": 6, "subject": "Task 6: Management command + validation + public shape", "status": "pending", "blockedBy": [2]}, - {"id": 7, "subject": "Task 7: CLI --transport flag", "status": "pending", "blockedBy": [6]}, - {"id": 8, "subject": "Task 8: Central UI transport selector", "status": "pending", "blockedBy": [6]}, - {"id": 9, "subject": "Task 9: Transport bundle carriage (Transport + OAuth2 drop fix)", "status": "pending", "blockedBy": [2]}, - {"id": 10, "subject": "Task 10: Docs sweep", "status": "pending", "blockedBy": [1, 2, 3, 4, 5, 6, 7, 8, 9]}, + {"id": 1, "subject": "Task 1: EmailTransport enum + parser", "status": "completed"}, + {"id": 2, "subject": "Task 2: SmtpConfiguration.Transport entity + EF mapping + migration", "status": "completed"}, + {"id": 3, "subject": "Task 3: EWS SOAP envelope builder + response parser", "status": "completed"}, + {"id": 4, "subject": "Task 4: EwsSoapMailSender + DI registration", "status": "completed", "blockedBy": [3]}, + {"id": 5, "subject": "Task 5: EmailNotificationDeliveryAdapter transport branch", "status": "completed", "blockedBy": [1, 2, 4]}, + {"id": 6, "subject": "Task 6: Management command + validation + public shape", "status": "completed", "blockedBy": [2]}, + {"id": 7, "subject": "Task 7: CLI --transport flag", "status": "completed", "blockedBy": [6]}, + {"id": 8, "subject": "Task 8: Central UI transport selector", "status": "completed", "blockedBy": [6]}, + {"id": 9, "subject": "Task 9: Transport bundle carriage (Transport + OAuth2 drop fix)", "status": "completed", "blockedBy": [2]}, + {"id": 10, "subject": "Task 10: Docs sweep", "status": "in_progress", "blockedBy": [1, 2, 3, 4, 5, 6, 7, 8, 9]}, {"id": 11, "subject": "Task 11: Integration verify + merge back", "status": "pending", "blockedBy": [10]} ], "lastUpdated": "2026-08-10" diff --git a/docs/plans/phase-7-integrations.md b/docs/plans/phase-7-integrations.md index 21edd954..3e04512b 100644 --- a/docs/plans/phase-7-integrations.md +++ b/docs/plans/phase-7-integrations.md @@ -463,7 +463,7 @@ Phase 7 is complete when: | # | Question | Context | Impact | Status | |---|----------|---------|--------|--------| -| Q12 | What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing? | Affects Notification Service OAuth2 implementation. | Phase 7. | Deferred — implement against Basic Auth first; OAuth2 tested when tenant available. | +| Q12 | What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing? | Affects Notification Service OAuth2 implementation. | Phase 7. | Closed 2026-08-10 as superseded — production mail is on-prem Exchange EWS (see [`docs/plans/2026-08-10-ews-email-transport-design.md`](2026-08-10-ews-email-transport-design.md)); OAuth2 SMTP path remains config-selectable but untested against a live M365 tenant (accepted). | (Existing question from questions.md — no new questions discovered.) diff --git a/docs/plans/questions.md b/docs/plans/questions.md index 9dc319e8..81a9f47a 100644 --- a/docs/plans/questions.md +++ b/docs/plans/questions.md @@ -10,7 +10,7 @@ | # | Question | Context | Impact | Status | |---|----------|---------|--------|--------| -| Q12 | What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing? | Affects Notification Service OAuth2 implementation. | Phase 7. | Deferred — won't be known during development. Implement against Basic Auth first; OAuth2 tested when tenant available. | +| Q12 | What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing? | Affects Notification Service OAuth2 implementation. | Phase 7. | Closed 2026-08-10 as superseded — production mail is on-prem Exchange EWS (see [`docs/plans/2026-08-10-ews-email-transport-design.md`](2026-08-10-ews-email-transport-design.md)); OAuth2 SMTP path remains config-selectable but untested against a live M365 tenant (accepted). | --- diff --git a/docs/requirements/Component-NotificationService.md b/docs/requirements/Component-NotificationService.md index 12555c6b..b3a138e5 100644 --- a/docs/requirements/Component-NotificationService.md +++ b/docs/requirements/Component-NotificationService.md @@ -53,6 +53,37 @@ The SMTP configuration is defined centrally and used by the central Email delive - **Connection timeout**: Maximum time to wait for SMTP connection (default: 30 seconds). - **Max concurrent connections**: Maximum simultaneous SMTP connections from the central cluster (default: 5). - **Retry settings**: Max retry count, fixed time between retries. The Notification Outbox reuses these for delivery retry of transient failures. +- **Transport**: The submission mechanism — `Smtp` (default) or `Ews`. Null/empty means `Smtp`, so every pre-existing row keeps its behavior. See EWS Transport below. + +### EWS Transport (on-prem Exchange) + +Email can be submitted through **Exchange Web Services** (SOAP over HTTPS) instead of SMTP, for networks where authenticated SMTP submission is not offered. The transport is chosen per configuration row via **Transport** = `Ews`; the SMTP/MailKit path is untouched and remains the default. Nothing changes for sites or scripts — delivery stays central-only and `Notify.To(...).Send(...)` behaves identically. Design rationale: [`docs/plans/2026-08-10-ews-email-transport-design.md`](../plans/2026-08-10-ews-email-transport-design.md). + +**Configuration model (field reuse).** No new entity or table — the same `SmtpConfiguration` row carries both transports: + +- **Host** — under `Ews`, the **full EWS endpoint URL**, an absolute `https://` URI (e.g. `https://mail.example.com/ews/exchange.asmx`), not a mail-server hostname. +- **Authentication mode** — must be `basic`. There is no OAuth2 or NTLM/Negotiate path on the EWS transport. +- **Credentials** — `username:password`, split on the **first** `:` only so a password may contain colons; the username may be domain-qualified (`domain\user`). Stored encrypted at rest via `EncryptedStringConverter`, as on the SMTP path, and projected away from every read path (`notification smtp list`, management responses, audit afterState). +- **From address**, **Connection timeout**, **Retry settings** — same meaning; the timeout becomes the per-request HTTP timeout. +- **Port**, **TLS mode**, **OAuth2 authority/scope**, **Max concurrent connections** — **unused** under `Ews`. The Central UI hides them and the CLI ignores them for this transport. + +**Auth posture.** HTTP Basic over HTTPS only, sent as an explicit per-request `Authorization` header (never set on the shared, pooled `HttpClient`, which would leak the credential between callers). HTTPS is enforced in **three** places — the management write gate, the delivery adapter, and again inside the sender as defense in depth — so a Basic credential can never leave the process over cleartext even if a row is edited around a higher layer. **Revisit trigger:** if Basic is disabled on the EWS virtual directory, an auth-mode knob (Negotiate/NTLM) becomes the documented follow-on; Linux containers would then also need `gss-ntlmssp` in the image. + +**Message semantics.** One `CreateItem` SOAP request per notification, `RequestServerVersion Exchange2013`, `MessageDisposition="SendOnly"`, recipients in **`BccRecipients` only**, body `BodyType="Text"`. BCC-only preserves the SMTP path's recipient handling (recipients cannot see each other — see Recipient Handling (Email) below). `SendOnly` deliberately leaves no Sent-Items copy: the central `Notifications` table is the audit record of what was sent, and a mailbox copy would grow unboundedly. All user content is XML-escaped. No SDK is used — a hand-rolled envelope over `HttpClient`, matching the no-SDK house pattern set by the SMS adapter. + +**Error classification (EWS).** Classification lives inside the EWS sender, which throws typed transient/permanent exceptions that the Email adapter maps to the same three-way `DeliveryOutcome` as SMTP (there is no standalone classifier type). **What Exchange said beats what HTTP reported**: a parsed response code decides the outcome even on an HTTP 500, and only an unparseable body falls back to the status code. + +- **Transient** (retry, then park after `MaxRetries`): connect/DNS/socket failures and request timeouts; HTTP 408/429/5xx on an unparseable body; and the availability-shaped response codes `ErrorServerBusy`, `ErrorInternalServerTransientError`, `ErrorTimeoutExpired`, `ErrorMailboxStoreUnavailable`, `ErrorInsufficientResources`. +- **Permanent** (park immediately): HTTP 401/403 (retrying a bad credential burns a domain account's lockout budget), 404/410/405 (wrong URL or endpoint), SOAP faults, recipient-shaped and schema response codes, and configuration defects found before the send (non-https or relative endpoint, non-Basic auth type, credentials not in `username:password` form). **Unclassified defaults to permanent**, matching the SMTP adapter's stance. +- Every surfaced message runs through `CredentialRedactor`, which masks both the packed `username:password` and its base64 Basic-auth encoding; attempts log the endpoint host and a recipient **count** only. + +**Response parsing.** EWS responses are parsed with DTD processing **prohibited** and no external entity resolver, so a hostile or malfunctioning endpoint cannot drive entity expansion or external-entity retrieval from the delivery path. A malformed body is reported as unparseable rather than throwing. + +**Validation layering.** The delivery adapter is the **authoritative** check — it re-validates the EWS shape at delivery time and parks the notification with a clear reason if the row is unusable, so no path can produce a silently broken send: + +- **Management write gate** (`ManagementActor`, covering the CLI and the management API) rejects an unknown transport, and for `Ews` requires an absolute https `Host`, `basic` auth mode, and `username:password` credentials. This catches the common mistake of flipping an existing SMTP row — whose credentials are a bare password — to `Ews`. +- **Central UI** `/notifications/smtp` writes through the notification repository directly rather than through the management gate, so the page mirrors the same https + Basic + credential-form rules client-side before saving. +- **Transport bundle import** deliberately does **not** gate EWS shape: an imported row is applied as data, and an unusable one surfaces as a parked notification with the adapter's error. Accepted — import is an environment-migration path where the operator remaps values anyway. ## SMS Configuration @@ -117,6 +148,7 @@ Each `Deliver(...)` call returns one of `success | transient failure | permanent - **Permanent failures** (SMTP 5xx permanent errors, e.g., mailbox not found): The Notification Outbox moves the row to `Parked` with the error in `LastError`. The notification will never deliver, and an operator can review or discard it on the Central UI Notification Outbox page. - Retries exhausted on a transient failure also result in a `Parked` row. - A script observes failures only by calling `Notify.Status(id)` and seeing a `Parked` status — not as a synchronous exception. +- The bullets above describe the SMTP transport. The EWS transport produces the same three outcomes from its own classification rules — see EWS Transport above. ### No Rate Limiting - No application-level rate limiting. If the delivery endpoint enforces sending limits (e.g., Microsoft 365 throttling or Twilio rate limits), those manifest as transient failures and are retried naturally by the Notification Outbox. diff --git a/docs/test_infra/test_infra_smtp.md b/docs/test_infra/test_infra_smtp.md index 59bec391..1f867d80 100644 --- a/docs/test_infra/test_infra_smtp.md +++ b/docs/test_infra/test_infra_smtp.md @@ -123,6 +123,8 @@ Use `--host` and `--port` to override SMTP defaults (localhost:1025), `--api` fo ## Notes - Mailpit does **not** support OAuth2 Client Credentials authentication. To test the OAuth2 code path, use a real Microsoft 365 tenant (see Q12 in `docs/plans/questions.md`). +- That OAuth2 tenant gap is **superseded for on-prem deployments** by the EWS email transport (`SmtpConfiguration.Transport = Ews`), which submits via Exchange Web Services over HTTPS with Basic auth and needs no tenant — see `docs/plans/2026-08-10-ews-email-transport-design.md`. Q12 is closed as superseded; the OAuth2 SMTP path stays config-selectable but untested against a live tenant. +- EWS has **no local test container**: its unit tests drive an in-process stub endpoint (envelope shape, Basic header, canned success / SOAP-fault / HTTP-error bodies), and the live gate against a real Exchange server is a **manual, one-off** exercise. Mailpit remains the SMTP-path harness and is unaffected. - To simulate SMTP failures for store-and-forward testing, stop the container: `docker compose stop smtp`. Restart with `docker compose start smtp`. - The web UI at `http://localhost:8025` provides real-time message inspection, search, and message source viewing. - No data persistence — messages are stored in a temporary database inside the container and lost on container removal. diff --git a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md index efc0f9c9..28c7b912 100644 --- a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md +++ b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md @@ -1302,7 +1302,7 @@ scadabridge --url notification smtp list Update the SMTP configuration. ```sh -scadabridge --url notification smtp update --id --server --port --auth-mode --from-address [--tls-mode ] [--credentials ] [--transport ] +scadabridge --url notification smtp update --id --server --port --auth-mode --from-address [--tls-mode ] [--credentials ] [--oauth2-authority ] [--oauth2-scope ] [--transport ] ``` | Option | Required | Description | @@ -1314,6 +1314,8 @@ scadabridge --url notification smtp update --id --server -- | `--from-address` | yes | Sender email address | | `--tls-mode` | no | TLS mode: `None`, `StartTLS`, or `SSL` (preserves existing if omitted) | | `--credentials` | no | SMTP credentials — `username:password` for Basic, or client secret for OAuth2 (preserves existing if omitted) | +| `--oauth2-authority` | no | OAuth2 token-endpoint URL — a `{tenant}` placeholder is substituted from the credential (preserves existing / Microsoft 365 default if omitted; OAuth2 auth mode only) | +| `--oauth2-scope` | no | OAuth2 scope requested from the token endpoint (preserves existing / Microsoft 365 default if omitted; OAuth2 auth mode only) | | `--transport` | no | Email transport: `Smtp` or `Ews` (preserves existing if omitted) | Under `--transport Ews`, `--server` carries the full EWS URL, `--auth-mode` must be `Basic`, diff --git a/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs b/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs index e39c57f6..17a912a2 100644 --- a/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs +++ b/src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs @@ -3439,6 +3439,9 @@ public sealed class BundleImporter : IBundleImporter target.RetryDelay = dto.RetryDelay; // Non-sensitive delivery metadata; null on a bundle written before these // were carried, which is the entity's "provider defaults / Smtp" state. + // Consequence, by design: a legacy (pre-Transport-field) bundle applied with + // Overwrite carries nulls here and so downgrades an EWS-configured target back + // to SMTP — visible to the operator as a Transport row in the import preview diff. target.OAuth2Authority = dto.OAuth2Authority; target.OAuth2Scope = dto.OAuth2Scope; target.Transport = dto.Transport;