ScadaBridge/docs/plans/2026-08-10-ews-email-transport-design.md

EWS Email Transport for the Notification Outbox — Design

Date: 2026-08-10 · Status: Implemented, merged to main; live gate PASS 2026-08-10 · Owner decisions captured below.

1. Context — the pending task this supersedes

The long-deferred "Office 365 / Exchange" email work is Q12 (docs/plans/questions.md:13, docs/plans/phase-7-integrations.md:466): "What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing?" The OAuth2 Client-Credentials SMTP path (OAuth2TokenService, XOAUTH2 in MailKitSmtpClientWrapper) shipped in Phase 7 but was never verified against a real tenant — Mailpit cannot speak OAuth2 (docs/test_infra/test_infra_smtp.md:125), and docs/deployment/production-checklist.md:46 still assumes OAuth2-over-587 as the production posture.

Q12's premise is now obsolete. The real mail infrastructure on the network where ScadaBridge runs is an on-prem Exchange 2013 server exposing EWS (SOAP over HTTPS) at https://webmail.zimmer.com/ews/exchange.asmx, with a service mailbox (ww_notify@zimmerbiomet.com, domain account nam\ww_notify). Dev/test credentials live in the untracked, git-ignored email_details.txt — they must never enter the repo, this doc, or a Transport bundle exported for another org.

Decision on Q12 (owner, 2026-08-10)

Close Q12 as superseded. The OAuth2 SMTP code path stays in place, config-selectable and unit-tested, but "untested against a real M365 tenant" becomes an accepted, documented state rather than a pending task. If email ever moves to Exchange Online, Q12's question revives.

2. Live evidence (probed 2026-08-10 from the dev Mac)

Probe	Result	Consequence
Unauthenticated GET /ews/exchange.asmx	401, WWW-Authenticate: Negotiate, NTLM, Basic — IIS 8.5, OWA 15.0.1497 (Exchange 2013)	Basic auth is enabled on the EWS vdir; NTLM is not required
Authenticated GET with HTTPS Basic (nam\ww_notify)	200	Basic works for the service account; the client is a plain HttpClient — cross-platform, macOS-testable, no GSSAPI/NTLM machinery
SMTP ports on the same host	25 open; 587/465 closed	Authenticated SMTP submission is not offered. Port 25 relay would be unauthenticated + receive-connector-IP-scoped — brittle, needs Exchange admin action per environment. Rejected as the transport (see Approach C)

3. Approaches considered

A. Transport mode inside the existing email pipeline — CHOSEN

Add a Transport discriminator (Smtp default / Ews) to the existing SmtpConfiguration row. EmailNotificationDeliveryAdapter branches after loading the config: the SMTP/MailKit path is untouched; the EWS path sends one SOAP CreateItem via a new seam. Everything around delivery — outbox lifecycle, retry/park, recipient resolution, KPIs, UI/CLI surface, bundle SecretsBlock travel — is reused unchanged. Trade-offs: smallest diff; one additive EF column; slight impurity of "SmtpConfiguration" naming now covering two transports (accepted — it is THE email config row, and renaming the entity/table would ripple through bundles and repos for cosmetics).

B. Separate EwsConfiguration entity + separate delivery adapter — rejected

A parallel entity/table/UI page/CLI noun and a second Email-type adapter. The outbox dispatcher keys adapters by NotificationType, so two Email adapters need a dispatcher change plus a which-config-wins rule; Transport bundles need a new artifact kind. Roughly double the surface for no behavioural gain.

C. No-code: point the existing SMTP adapter at Exchange — rejected on evidence

Would be zero code, but the probe shows authenticated submission (587/465) is closed. Port-25 relay is anonymous, scoped by source IP on a receive connector, and an Exchange admin change per environment — an unauthenticated, brittle dependency the moment containers move hosts.

EWS client style (owner decision)

Hand-rolled SOAP envelope over HttpClient — no SDK, matching the house pattern set by SmsNotificationDeliveryAdapter (Twilio REST, no SDK). The Microsoft EWS Managed API is archived; the community fork is a heavyweight dependency for what is one fixed CreateItem call (~200 lines including classification).

4. Design

4.1 Configuration model

• SmtpConfiguration (Commons POCO + SmtpConfigurations table) gains one additive column: Transport (string, default "Smtp"; parsed case-insensitively like TlsMode). One EF migration; existing rows keep SMTP semantics with no data change.
• Field reuse under Transport = "Ews":
  • Host — the full EWS endpoint URL (absolute https:// URI, validated).
  • AuthType — must be basic; Credentials stays username:password where username may be domain\user (split on the first : only, as today). Same storage treatment as SMTP credentials: SmtpConfiguration.Credentials is encrypted at rest via EncryptedStringConverter (wired in ScadaBridgeDbContext.ApplySecretColumnEncryption, ConfigurationDatabase), exactly like SmsConfiguration.AuthToken — there is no asymmetry between them. It is additionally projected away from all read paths by ManagementActor's credential-free projection, and CredentialRedactor.Scrub runs on every error message.
  • FromAddress, MaxRetries, RetryDelay, ConnectionTimeoutSeconds — same meaning (timeout becomes the HttpClient request timeout).
  • Port, TlsMode, OAuth2Authority, OAuth2Scope, MaxConcurrentConnections — unused for EWS; validation requires them unset/ignored and the UI hides them.
• Validation (in HandleUpdateSmtpConfig + options validator, eager per §1.5 conventions): unknown Transport rejected; Ews requires absolute https Host, basic auth, non-empty credentials; Smtp validation unchanged.

4.2 Delivery path

• New seam in the NotificationService project: IEwsMailSender with one method SendAsync(EwsSendRequest, CancellationToken) (endpoint, credential, from, bcc list, subject, plain-text body). Implementation EwsSoapMailSender:
  • IHttpClientFactory named client; explicit Authorization: Basic header (deterministic, proven live) rather than handler-negotiated auth. If Basic is ever disabled in a hardening pass, an EwsAuthMode knob (Negotiate/NTLM via HttpClientHandler.Credentials) is the documented follow-on — not built now (YAGNI), and noted that NTLM-on-Linux containers would then need gss-ntlmssp in the image.
  • One CreateItem SOAP envelope, RequestServerVersion Exchange2013, MessageDisposition="SendOnly" (no Sent-Items copy — the Notifications table is the audit record of what was sent; a mailbox copy would just grow unboundedly). Recipients go in BccRecipients only, preserving the SMTP path's BCC semantics (recipients can't see each other). All user content XML-escaped; body BodyType="Text" (plain text, as today).
• EmailNotificationDeliveryAdapter.DeliverAsync branches on the parsed transport after the existing list-resolution/config/address-validation steps (all shared): Smtp → current SendAsync; Ews → IEwsMailSender. Outcome mapping stays three-way DeliveryOutcome.

4.3 Error classification (mirrors SmtpErrorClassifier / ESG conventions)

• Transient (retry → park after MaxRetries): connect/DNS/socket/timeout failures; HTTP 5xx/408/429; response/fault codes that are load/availability-shaped (ErrorServerBusy, ErrorInternalServerTransientError, ErrorTimeoutExpired, ErrorMailboxStoreUnavailable, ErrorInsufficientResources).
• Permanent (park immediately): HTTP 401/403 (credential/authorization — retrying burns lockout budget on a domain account), 404/410/405 (wrong URL or endpoint), SOAP schema faults, recipient-shaped response codes (ErrorInvalidRecipients, ErrorMessageSizeExceeded), malformed-config findings (bad URL, bad credential form) — same "unclassified defaults to permanent" stance the SMTP adapter takes.
• As built: there is no standalone EwsErrorClassifier type. Classification lives inside EwsSoapMailSender, which throws typed EwsTransientException / EwsPermanentException that the adapter maps to DeliveryOutcome; the pure EwsResponseParser stays judgement-free, reporting only the response shape and code (and parsing with DTD processing prohibited). A parsed response code beats the HTTP status — only an unparseable body falls back to the status code. Every surfaced message runs through CredentialRedactor.Scrub, plus a mask of the base64 Basic-auth value.

4.4 Management surfaces

• CLI: notification smtp update gains --transport smtp|ews (existing --host, --auth-type, --credentials, retry flags reused). notification smtp list shows the transport; credential-free projection unchanged.
• Central UI /notifications/smtp: a Transport selector; EWS mode shows URL + username + password inputs and hides Port/TLS/OAuth2 inputs (which remain SMTP-only, OAuth2 inputs remaining auth-type-gated as today). Admin-only, as today.
• Transport bundles: SmtpConfiguration already travels in the encrypted SecretsBlock; the new field rides along additively (schemaVersion additive rules). The existing round-trip guard (arch-review T8 style) is extended to cover Transport so export/import can't silently drop it. Discovered during implementation: OAuth2Authority and OAuth2Scope (shipped 2026-07) were already being silently dropped by bundle export/import — SmtpConfigDto never carried them. Fixed in the same slice as Transport (DTO, serializer, BundleImporter.ApplySmtpFields, ArtifactDiff), and the round-trip guard reseeded so all three fields are now pinned.

4.5 Testing (owner decision: fake stub + live gate)

• Unit: EwsSoapMailSender against a stubbed HttpMessageHandler (in-process fake EWS endpoint) asserting the Basic header, envelope shape (BCC-only, SendOnly, escaping) and driving canned CreateItemResponse success / SOAP-fault / HTTP-error bodies through the sender's classification (see §4.3 — no standalone classifier type); adapter-level tests for the transport branch and outcome mapping; validator tests for the new config rules. All macOS-runnable — no NTLM, no network.
• Live gate (manual, one-off): from the dev Mac, configure Transport=Ews with the dev/test credentials (from the untracked email_details.txt) against webmail.zimmer.com, send to a single-recipient test list, verify Delivered status + received mail + a permanent-classification case (bad password on a throwaway config — mindful of domain lockout policy, one attempt only). Recorded as a PASS note in this doc when run.
• Mailpit-based SMTP tests are untouched (the SMTP path is untouched).

4.6 Documentation updates that travel with the implementation

• Component-NotificationService.md: EWS transport section (config model, classification, BCC/SendOnly semantics, auth posture).
• docs/plans/questions.md Q12 + phase-7-integrations.md Q12 row: closed as superseded, pointing here.
• docs/deployment/production-checklist.md: EWS variant (outbound HTTPS 443 to the Exchange CAS; service-account password rotation note) alongside the SMTP items.
• docs/test_infra/test_infra_smtp.md: note that the OAuth2 gap is superseded by EWS for on-prem deployments.

Live gate result (2026-08-10) — PASS

Run on the docker rig, rebuilt from main with the EWS image. The EWS configuration was created through the new Central UI transport selector (its own first live exercise), pointing at the real on-prem endpoint with the dev service mailbox. One SoakNotify tick was enabled for ~9 s; the resulting notification delivered on the first attempt (Status=Delivered, RetryCount=0, resolved target = the tester's real mailbox) through https://webmail.zimmer.com/ews/exchange.asmx, and the message was received. Pre-existing parked rows (from the no-config era) were untouched. The rig was then restored to baseline: SMTP config row removed, notification list recipient restored, soak instance disabled. Note: the rig's LDAP login was found broken by the SEC-36 GLAuth service-account rotation (stale serviceaccount123 in docker-compose); the gate ran under the Development-only DisableLogin switch, left active pending the rotated credential.

5. Explicitly out of scope

• Removing the SMTP/OAuth2 path (stays config-selectable; Approach "additional transport" was the owner's choice).
• NTLM/Negotiate auth mode (documented follow-on trigger: Basic disabled on the EWS vdir).
• HTML bodies, attachments, per-recipient sends, Sent-Items copies.
• Site-side anything — notification delivery remains central-only.

6. Follow-ups (from execution reviews)

Raised while implementing; none blocking, none scheduled here.

1. Catch-chain duplication in EmailNotificationDeliveryAdapter. The SMTP and EWS branches each carry a near-identical permanent / caller-cancel / transient / unclassified catch chain differing only in exception types and log wording. Extract a shared helper before a third transport is added.
2. Bundle-import shape validation parity. Import applies SmtpConfigDto fields as data and does not run the EWS shape gate that ManagementActor (CLI/API) and the Central UI apply. The delivery adapter is authoritative and parks an unusable row with a clear reason, so this is accepted for now rather than duplicating the gate a fourth time.
3. CredentialRedactor.MinSecretLength = 12. A standalone password shorter than that is not scrubbed on its own; EWS messages are still covered by the packed username:password and base64 Basic-auth scrubs, which comfortably exceed the floor. Revisit if a code path ever surfaces a bare short password.
4. Central UI SMTP page bypasses audit. SmtpConfiguration.razor writes through INotificationRepository directly (pre-existing pattern), so a UI change to Transport or the EWS credential produces no audit row, while the same change via CLI/API is audited by ManagementActor. Made more consequential by EWS; route the page through UpdateSmtpConfigCommand (or add a direct audit call) in a follow-up.