From 2dc919e8a42a5a0eed258c52fe9d94b21cf8e3f5 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 10 Aug 2026 16:22:31 -0400 Subject: [PATCH] docs(ldap): NAM AD setup notes + live TLS probe results; backup-DC failover design (Auth 0.2.0) --- ...26-08-10-ldap-backup-dc-failover-design.md | 139 ++++++++++++++++++ ldap_setup.md | 104 +++++++++++++ 2 files changed, 243 insertions(+) create mode 100644 docs/plans/2026-08-10-ldap-backup-dc-failover-design.md create mode 100644 ldap_setup.md diff --git a/docs/plans/2026-08-10-ldap-backup-dc-failover-design.md b/docs/plans/2026-08-10-ldap-backup-dc-failover-design.md new file mode 100644 index 0000000..7550ba1 --- /dev/null +++ b/docs/plans/2026-08-10-ldap-backup-dc-failover-design.md @@ -0,0 +1,139 @@ +# LDAP backup-DC failover — design (`ZB.MOM.WW.Auth` 0.2.0) + +**Date:** 2026-08-10 +**Status:** Approved +**Motivation:** Production servers at ZB authenticate against the Zimmer NAM AD +([`ldap_setup.md`](../../ldap_setup.md)), which has two domain controllers +(`uswap-vdc-nam1` / `uswap-vdc-nam2`). `ZB.MOM.WW.Auth.Ldap` takes a single +`Server`, so a primary-DC outage blocks every interactive login family-wide +(API keys unaffected). The working Ignition gateway has primary + secondary DC +failover; this brings the shared library to parity on that axis. + +## Decisions (made with the user, 2026-08-10) + +1. **Sticky preference** failover — try the last-known-good DC first; on failure + walk the ordered list and remember whichever worked. Mirrors the Secrets + 0.5.0 `FallbackEndpoints` sticky/no-flapping precedent. Rejected: + always-primary-first (pays the full connection timeout on every login during + a primary outage), sticky + probe-back (YAGNI for two DCs). +2. **Failover-only** — no Ignition-style credential cache. A both-DCs-down event + is rare and correlated with wider outages; API keys already survive it; a + credential cache adds credential-at-rest surface. Possible future follow-up. +3. **Approach A: service-level loop** — the walk lives in `LdapAuthService`, + which already owns failure classification (user-fault vs system-fault). + Rejected: connection-factory-level (only sees `Connect`; a DC that accepts + TCP+TLS but fails service-bind/search would never fail over), DNS-level + `nam.zimmer.com` (DC certs' SANs carry only their own FQDN → LDAPS hostname + validation breaks; no stickiness or timeout control). + +## Probe evidence (2026-08-10, recorded in `ldap_setup.md`) + +Both DCs serve LDAPS (636, TLSv1.3) and StartTLS (389); anonymous rootDSE over +LDAPS returns `result: 0 Success` on both. Certs: per-host FQDN SANs, issued by +`zimmer-issueca1` ← `zimmer-rootca` (internal CA, root not served), valid to +2027-03. **Caveat: both certs are RSA 1024-bit** — OpenSSL at SECLEVEL=2 +(Debian-based .NET containers) rejects the handshake (`EE certificate key too +weak`); Windows/Schannel hosts accept. Container deployments need a lowered +SECLEVEL, a validation-callback path, or (right fix) a reissued 2048-bit DC +cert template. + +## Config surface + +One additive field on `LdapOptions` (`ZB.MOM.WW.Auth.Abstractions`): + +```csharp +/// Ordered fallback LDAP endpoints, "host" or "host:port"; a bare host uses . +public IReadOnlyList FallbackServers { get; init; } = Array.Empty(); +``` + +```jsonc +"Ldap": { + "Server": "uswap-vdc-nam1.nam.zimmer.com", + "Port": 636, + "Transport": "Ldaps", + "FallbackServers": [ "uswap-vdc-nam2.nam.zimmer.com" ] +} +``` + +- Empty (the default) ⇒ behavior byte-identical to today: single endpoint, no + loop, no new state. +- `ServerCertificateValidationCallback` and `AllowInsecure` apply to every + endpoint (one CA family — verified by the probe). +- **Validator** (`LdapOptionsValidator`, only when `Enabled`): each entry must + have a non-empty host; an explicit port must parse as 1–65535. Malformed + entries fail boot with a field-naming message in the validator's existing + style. Duplicates are permitted (harmless; de-duplication is not the + validator's job). + +## Failover semantics + +Inside `LdapAuthService.Authenticate`, an endpoint walk over an internal +`LdapEndpoint(string Host, int Port)` list = `Server:Port` + parsed +`FallbackServers`, in declared order: + +- **Walk order:** sticky-preferred endpoint first, then the remaining endpoints + in declared order (skipping the preferred). On success at endpoint *i*, + preference := *i*. +- **Only system-side failures advance the walk:** connect/TLS-upgrade failure, + service-account bind failure, search failure, and any unexpected exception + inside an attempt. **User-side results return immediately, never fail over:** + `UserNotFound`, `AmbiguousUser`, `BadCredentials` (user bind), + `GroupLookupFailed` — those are answers from a healthy directory. +- **Exhaustion:** all endpoints failed system-side ⇒ return the last failure + (maps to `ServiceAccountBindFailed`, as today — no `LdapAuthFailure` enum + change, no `Abstractions` contract change beyond the new option). +- **Fail-closed contract preserved:** `AuthenticateAsync` still never throws; + each per-endpoint attempt is individually guarded. +- Documented trade-offs: + - A genuinely wrong service-account password tries every DC per login (fails + fast at bind — no timeout cost; 2 endpoints in practice). + - Worst-case login latency while the preferred DC is down = + `ConnectionTimeoutMs` × walk position, paid once — stickiness then moves + preference to the survivor. + - Sticky means the primary stays unused after recovery until the current + preferred fails; when it does, the walk tries declared order again, so + preference drifts back to the primary naturally. + +## Stickiness state + +`volatile int _preferredIndex` on the singleton `LdapAuthService` (registered +`TryAddSingleton` in `ZB.MOM.WW.Auth.AspNetCore`). Benign races tolerated — +worst case is one redundant failed attempt. No cross-process or persisted +state. + +## Observability + +New optional `ILogger?` constructor parameter (null default — +existing constructors keep compiling; adds only +`Microsoft.Extensions.Logging.Abstractions` to `ZB.MOM.WW.Auth.Ldap`). The +AspNetCore registration passes the host logger. Log points: + +- **Warning** per failover hop: endpoint, stage that failed, next endpoint. +- **Information** on sticky-preference change (old → new endpoint). +- Nothing on the happy path; never log credentials or passwords. + +Rationale: silent failover hides a dead primary DC indefinitely — the feature +working correctly must be visible to operators. + +## Testing + +- **Unit (FakeLdapConnection seam):** failover triggered by each system-side + stage (connect, service bind, search, unexpected exception); NO failover on + each user-side result; stickiness updates on success and drifts back after + the preferred fails; exhaustion returns `ServiceAccountBindFailed`; empty + `FallbackServers` short-circuits to today's exact behavior; endpoint parsing + (`host`, `host:port`, malformed). +- **Validator tests:** new cases in `LdapOptionsValidatorTests` style. +- **Integration (env-gated GLAuth suite):** primary pointed at a closed local + port + fallback at the real GLAuth ⇒ login succeeds, preference sticks, and a + second login does not retry the dead primary. Fast offline too — a closed + local port refuses immediately. + +## Versioning / rollout + +- Auth family `0.1.5` → **`0.2.0`** (additive feature; the four packages + version together). Publish all 4 to the Gitea feed, restore-verify. +- **Zero consumer code changes** — the four apps adopt by config alone at ZB + deploy time. Bumping the apps' pins to 0.2.0 is a follow-up family-alignment + task, not a prerequisite. +- Dev/GLAuth stacks are unaffected (no `FallbackServers` configured). diff --git a/ldap_setup.md b/ldap_setup.md new file mode 100644 index 0000000..a02aab8 --- /dev/null +++ b/ldap_setup.md @@ -0,0 +1,104 @@ +# Zimmer NAM — LDAP / Active Directory Auth Setup + +Reference for configuring services to authenticate against the Zimmer NAM domain. +Extracted from the production Ignition 8.1 gateway `Ignition-IGNTN-APP-VP01` +(`config.idb`, `NAM` AD-Hybrid user source), audited 2026-08-05. Full audit: +`~/Desktop/config-idb-users-groups.md`. + +## Directory / connection + +| Setting | Value | +|---------|-------| +| Domain | `nam.zimmer.com` | +| Primary DC | `uswap-vdc-nam1:389` | +| Secondary DC | `uswap-vdc-nam2:389` | +| Encryption | **None** — plain LDAP on 389 (no LDAPS/StartTLS in the Ignition config) | +| SASL | `DIGEST-MD5`, `CRAM-MD5` offered; QoP `auth-conf,auth-int,auth`; strength `high,medium,low`; no mutual auth | +| Referrals | FOLLOW | +| Read timeout | 60 000 ms | +| Paging | 1000 entries/page | +| Anonymous bind | Disallowed | + +## Bind identity (UPN, not DN) + +- Service account: `svc_igntnapp` +- Binds by **UPN**: `svc_igntnapp@nam.zimmer.com` — there is no bind DN/CN anywhere + in the working config; end users also authenticate as `@nam.zimmer.com` + (automatic suffix). +- Password: held encrypted by the gateway; obtain from the credential owner — never + commit it (in this repo's services use a `${secret:NAME}` reference). + +## User lookup + +| Setting | Value | +|---------|-------| +| Search base | domain root `DC=nam,DC=zimmer,DC=com` (Ignition field left empty) | +| User search filter | `(&(objectClass=user)(sAMAccountName={0}))` | +| User enumeration filter | `(&(objectClass=user)(!(objectClass=computer)))` | +| Username attribute | `sAMAccountName` | +| Display name | `name` | +| Email | `mail` | +| Phone / SMS | `telephoneNumber` / `mobile` | + +## Groups / roles + +**The working setup does not use LDAP groups at all.** The Ignition profile is +AD-Hybrid: AD answers *who you are*, while *what you may do* (roles +Administrator, Designer, Engineer, Maintenance, Management, Mfg_Tech, Operator) +is assigned manually per user inside the application. AD security-group +membership grants nothing. + +Implication for new services: don't assume a `memberOf`-based authorization +scheme exists to mirror — there is no established group→role convention in this +domain's SCADA stack. Either keep app-local role assignment (matching the +existing pattern) or define new AD groups deliberately; standard AD attributes +(`memberOf`, filter `(&(objectClass=group)(member={0}))`) are available if we +choose to. + +## Behavioral notes from the working config + +- Users are **populated on demand**: a local record is created on first + successful AD login (with no roles until granted). +- Failover: soft failover to the local internal user source, 60 s credential-cache + validation — cached logins keep working through a DC outage. +- Lockout: 5 failed attempts / 15-minute window. + +## Cautions + +- Port 389 without TLS means simple binds transit cleartext unless SASL + confidentiality is negotiated — for new .NET services prefer LDAPS (636) or + StartTLS against the same DCs. + +## TLS probe results (verified 2026-08-10, from the dev Mac) + +Both DCs were live-probed; **LDAPS and StartTLS both work on both DCs.** + +| Check | uswap-vdc-nam1 | uswap-vdc-nam2 | +|---|---|---| +| Ports open | 389, 636, 3268, 3269 | 389, 636, 3268, 3269 | +| LDAPS (636) handshake | TLSv1.3 OK | TLSv1.3 OK | +| StartTLS on 389 | OK | OK | +| Anonymous rootDSE over LDAPS | `result: 0 Success` | `result: 0 Success` | +| Cert subject/SAN | `USWAP-VDC-NAM1.nam.zimmer.com` (matches FQDN) | `USWAP-VDC-NAM2.nam.zimmer.com` (matches FQDN) | +| Cert validity | 2026-03-21 → 2027-03-21 | 2026-03-20 → 2027-03-20 | +| Chain | leaf ← `zimmer-issueca1` ← `zimmer-rootca` (intermediate served, root not) | same | +| **Key size** | **RSA 1024-bit** (sha256 sig) | **RSA 1024-bit** (sha256 sig) | + +Implications for new .NET services: + +- **Use LDAPS 636** (or StartTLS) — no reason to accept cleartext binds. +- **Trust:** `zimmer-rootca` is an internal CA, not in default OS/container trust + stores. Either install the root (+ intermediate) into the host/container trust + store, or pin the chain via `LdapOptions.ServerCertificateValidationCallback` + (code-only seam in `ZB.MOM.WW.Auth.Ldap`). +- **⚠️ RSA 1024-bit DC certs:** OpenSSL at default security level (SECLEVEL=2 — + what Debian-based .NET containers ship) **rejects 1024-bit keys** + (`EE certificate key too weak`, observed in this probe), so a Linux/container + client may fail the TLS handshake even with the CA trusted. Windows/Schannel + hosts accept them. Options: get the DC cert template reissued at 2048+ (right + fix, needs AD PKI owner), or lower SECLEVEL / use a custom validation callback + in affected containers (workaround). Certs expire 2027-03 — a reissue + opportunity. +- Hostnames resolve from the dev network: nam1 = `10.201.216.2`, + nam2 = `10.201.138.78`. +