From b2ea9c6c746c110628d6b9fe32b2c8f6cbb1e6eb Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 10 Aug 2026 05:56:41 -0400 Subject: [PATCH] docs(notifications): design EWS email transport for the outbox; close Q12 as superseded MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On-prem Exchange 2013 EWS (Basic over HTTPS, live-probed) becomes a second selectable email transport beside SMTP: additive SmtpConfiguration.Transport discriminator, hand-rolled CreateItem SOAP sender (no SDK, BCC-only, SendOnly), EwsErrorClassifier mirroring the SMTP transient/permanent split, CLI/UI transport selector, fake-EWS unit stub + one-off live gate. The pending O365 SMTP-OAuth2 verification (Q12) is superseded — real mail infra is on-prem EWS. Design only; no implementation. Also gitignore the untracked dev-credential file email_details.txt. --- .gitignore | 1 + .../2026-08-10-ews-email-transport-design.md | 159 ++++++++++++++++++ 2 files changed, 160 insertions(+) create mode 100644 docs/plans/2026-08-10-ews-email-transport-design.md diff --git a/.gitignore b/.gitignore index 70761ed8..b7c63abe 100644 --- a/.gitignore +++ b/.gitignore @@ -55,3 +55,4 @@ docker-env2/*/data/ # Sister-project deployment artifacts (not part of this solution) /deploy/ +email_details.txt diff --git a/docs/plans/2026-08-10-ews-email-transport-design.md b/docs/plans/2026-08-10-ews-email-transport-design.md new file mode 100644 index 00000000..ee1f525c --- /dev/null +++ b/docs/plans/2026-08-10-ews-email-transport-design.md @@ -0,0 +1,159 @@ +# EWS Email Transport for the Notification Outbox — Design + +**Date:** 2026-08-10 · **Status:** Designed, not implemented · **Owner decisions captured below.** + +## 1. Context — the pending task this supersedes + +The long-deferred "Office 365 / Exchange" email work is **Q12** (`docs/plans/questions.md:13`, +`docs/plans/phase-7-integrations.md:466`): *"What Microsoft 365 tenant/app registration is +available for SMTP OAuth2 testing?"* The OAuth2 Client-Credentials SMTP path +(`OAuth2TokenService`, XOAUTH2 in `MailKitSmtpClientWrapper`) shipped in Phase 7 but was never +verified against a real tenant — Mailpit cannot speak OAuth2 (`docs/test_infra/test_infra_smtp.md:125`), +and `docs/deployment/production-checklist.md:46` still assumes OAuth2-over-587 as the production +posture. + +**Q12's premise is now obsolete.** The real mail infrastructure on the network where ScadaBridge +runs is an **on-prem Exchange 2013** server exposing **EWS** (SOAP over HTTPS) at +`https://webmail.zimmer.com/ews/exchange.asmx`, with a service mailbox +(`ww_notify@zimmerbiomet.com`, domain account `nam\ww_notify`). Dev/test credentials live in the +**untracked, git-ignored** `email_details.txt` — they must never enter the repo, this doc, or a +Transport bundle exported for another org. + +### Decision on Q12 (owner, 2026-08-10) +**Close Q12 as superseded.** The OAuth2 SMTP code path stays in place, config-selectable and +unit-tested, but "untested against a real M365 tenant" becomes an accepted, documented state +rather than a pending task. If email ever moves to Exchange Online, Q12's question revives. + +## 2. Live evidence (probed 2026-08-10 from the dev Mac) + +| Probe | Result | Consequence | +|---|---|---| +| Unauthenticated `GET /ews/exchange.asmx` | `401`, `WWW-Authenticate: Negotiate, NTLM, Basic` — IIS 8.5, OWA 15.0.1497 (Exchange 2013) | Basic auth is enabled on the EWS vdir; **NTLM is not required** | +| Authenticated GET with HTTPS Basic (`nam\ww_notify`) | `200` | Basic works for the service account; the client is a plain `HttpClient` — cross-platform, macOS-testable, no GSSAPI/NTLM machinery | +| SMTP ports on the same host | 25 open; **587/465 closed** | Authenticated SMTP submission is not offered. Port 25 relay would be unauthenticated + receive-connector-IP-scoped — brittle, needs Exchange admin action per environment. Rejected as the transport (see Approach C) | + +## 3. Approaches considered + +### A. Transport mode inside the existing email pipeline — **CHOSEN** +Add a `Transport` discriminator (`Smtp` default / `Ews`) to the existing `SmtpConfiguration` +row. `EmailNotificationDeliveryAdapter` branches after loading the config: the SMTP/MailKit path +is untouched; the EWS path sends one SOAP `CreateItem` via a new seam. Everything around +delivery — outbox lifecycle, retry/park, recipient resolution, KPIs, UI/CLI surface, bundle +SecretsBlock travel — is reused unchanged. +*Trade-offs:* smallest diff; one additive EF column; slight impurity of "SmtpConfiguration" +naming now covering two transports (accepted — it is THE email config row, and renaming the +entity/table would ripple through bundles and repos for cosmetics). + +### B. Separate `EwsConfiguration` entity + separate delivery adapter — rejected +A parallel entity/table/UI page/CLI noun and a second Email-type adapter. The outbox dispatcher +keys adapters by `NotificationType`, so two Email adapters need a dispatcher change plus a +which-config-wins rule; Transport bundles need a new artifact kind. Roughly double the surface +for no behavioural gain. + +### C. No-code: point the existing SMTP adapter at Exchange — rejected on evidence +Would be zero code, but the probe shows authenticated submission (587/465) is closed. Port-25 +relay is anonymous, scoped by source IP on a receive connector, and an Exchange admin change per +environment — an unauthenticated, brittle dependency the moment containers move hosts. + +### EWS client style (owner decision) +**Hand-rolled SOAP envelope over `HttpClient` — no SDK**, matching the house pattern set by +`SmsNotificationDeliveryAdapter` (Twilio REST, no SDK). The Microsoft EWS Managed API is +archived; the community fork is a heavyweight dependency for what is one fixed `CreateItem` +call (~200 lines including classification). + +## 4. Design + +### 4.1 Configuration model +- `SmtpConfiguration` (Commons POCO + `SmtpConfigurations` table) gains one **additive** column: + `Transport` (`string`, default `"Smtp"`; parsed case-insensitively like `TlsMode`). One EF + migration; existing rows keep SMTP semantics with no data change. +- Field reuse under `Transport = "Ews"`: + - `Host` — the **full EWS endpoint URL** (absolute `https://` URI, validated). + - `AuthType` — must be `basic`; `Credentials` stays `username:password` where username may be + `domain\user` (split on the **first** `:` only, as today). Same storage treatment as SMTP + credentials (persisted server-side, projected away from all read paths by + `ManagementActor`'s credential-free projection; `CredentialRedactor.Scrub` on every error + message). The pre-existing asymmetry with `SmsConfiguration`'s Data-Protection-encrypted + `AuthToken` is noted but out of scope here. + - `FromAddress`, `MaxRetries`, `RetryDelay`, `ConnectionTimeoutSeconds` — same meaning + (timeout becomes the `HttpClient` request timeout). + - `Port`, `TlsMode`, `OAuth2Authority`, `OAuth2Scope`, `MaxConcurrentConnections` — unused for + EWS; validation requires them unset/ignored and the UI hides them. +- Validation (in `HandleUpdateSmtpConfig` + options validator, eager per §1.5 conventions): + unknown `Transport` rejected; `Ews` requires absolute https `Host`, `basic` auth, non-empty + credentials; `Smtp` validation unchanged. + +### 4.2 Delivery path +- New seam in the NotificationService project: `IEwsMailSender` with one method + `SendAsync(EwsSendRequest, CancellationToken)` (endpoint, credential, from, bcc list, subject, + plain-text body). Implementation `EwsSoapMailSender`: + - `IHttpClientFactory` named client; **explicit `Authorization: Basic` header** (deterministic, + proven live) rather than handler-negotiated auth. If Basic is ever disabled in a hardening + pass, an `EwsAuthMode` knob (Negotiate/NTLM via `HttpClientHandler.Credentials`) is the + documented follow-on — not built now (YAGNI), and noted that NTLM-on-Linux containers would + then need `gss-ntlmssp` in the image. + - One `CreateItem` SOAP envelope, `RequestServerVersion Exchange2013`, + `MessageDisposition="SendOnly"` (no Sent-Items copy — the `Notifications` table is the audit + record of what was sent; a mailbox copy would just grow unboundedly). Recipients go in + `BccRecipients` only, preserving the SMTP path's BCC semantics (recipients can't see each + other). All user content XML-escaped; body `BodyType="Text"` (plain text, as today). +- `EmailNotificationDeliveryAdapter.DeliverAsync` branches on the parsed transport after the + existing list-resolution/config/address-validation steps (all shared): `Smtp` → current + `SendAsync`; `Ews` → `IEwsMailSender`. Outcome mapping stays three-way `DeliveryOutcome`. + +### 4.3 Error classification (mirrors `SmtpErrorClassifier` / ESG conventions) +- **Transient** (retry → park after `MaxRetries`): connect/DNS/socket/timeout failures; HTTP + 5xx/408/429; SOAP fault codes that are load/availability-shaped (`ErrorServerBusy`, + `ErrorInternalServerTransientError`, `ErrorTimeoutExpired`, `ErrorMailboxStoreUnavailable`). +- **Permanent** (park immediately): HTTP 401/403 (credential/authorization — retrying burns + lockout budget on a domain account), 404 (wrong URL), SOAP schema faults, recipient-shaped + response codes (`ErrorInvalidRecipients`, `ErrorMessageSizeExceeded`), malformed-config + findings (bad URL, bad credential form) — same "unclassified defaults to permanent" stance the + SMTP adapter takes. +- New pure `EwsErrorClassifier` alongside `SmtpErrorClassifier`; every surfaced message runs + through `CredentialRedactor.Scrub`. + +### 4.4 Management surfaces +- **CLI:** `notification smtp update` gains `--transport smtp|ews` (existing `--host`, + `--auth-type`, `--credentials`, retry flags reused). `notification smtp list` shows the + transport; credential-free projection unchanged. +- **Central UI `/notifications/smtp`:** a Transport selector; EWS mode shows URL + username + + password inputs and hides Port/TLS/OAuth2 inputs (which remain SMTP-only, OAuth2 inputs + remaining auth-type-gated as today). Admin-only, as today. +- **Transport bundles:** `SmtpConfiguration` already travels in the encrypted SecretsBlock; the + new field rides along additively (`schemaVersion` additive rules). The existing round-trip + guard (arch-review T8 style) is extended to cover `Transport` so export/import can't silently + drop it. + +### 4.5 Testing (owner decision: fake stub + live gate) +- **Unit:** `EwsSoapMailSender` against an in-process fake EWS endpoint (Kestrel `TestServer`) + asserting the Basic header, envelope shape (BCC-only, SendOnly, escaping) and driving canned + `CreateItemResponse` success / SOAP-fault / HTTP-error bodies through `EwsErrorClassifier`; + adapter-level tests for the transport branch and outcome mapping; validator tests for the new + config rules. All macOS-runnable — no NTLM, no network. +- **Live gate (manual, one-off):** from the dev Mac, configure `Transport=Ews` with the + dev/test credentials (from the untracked `email_details.txt`) against + `webmail.zimmer.com`, send to a single-recipient test list, verify Delivered status + + received mail + a permanent-classification case (bad password on a throwaway config — + mindful of domain lockout policy, one attempt only). Recorded as a PASS note in this doc when + run. +- Mailpit-based SMTP tests are untouched (the SMTP path is untouched). + +### 4.6 Documentation updates that travel with the implementation +- `Component-NotificationService.md`: EWS transport section (config model, classification, + BCC/SendOnly semantics, auth posture). +- `docs/plans/questions.md` Q12 + `phase-7-integrations.md` Q12 row: closed as superseded, + pointing here. +- `docs/deployment/production-checklist.md`: EWS variant (outbound HTTPS 443 to the Exchange + CAS; service-account password rotation note) alongside the SMTP items. +- `docs/test_infra/test_infra_smtp.md`: note that the OAuth2 gap is superseded by EWS for + on-prem deployments. + +## 5. Explicitly out of scope +- Removing the SMTP/OAuth2 path (stays config-selectable; Approach "additional transport" was + the owner's choice). +- NTLM/Negotiate auth mode (documented follow-on trigger: Basic disabled on the EWS vdir). +- Encrypting `SmtpConfiguration.Credentials` at rest (pre-existing posture, tracked separately + if desired). +- HTML bodies, attachments, per-recipient sends, Sent-Items copies. +- Site-side anything — notification delivery remains central-only.