docs(ldap): NAM AD setup notes + live TLS probe results; backup-DC failover design (Auth 0.2.0)
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,139 @@
|
||||
# LDAP backup-DC failover — design (`ZB.MOM.WW.Auth` 0.2.0)
|
||||
|
||||
**Date:** 2026-08-10
|
||||
**Status:** Approved
|
||||
**Motivation:** Production servers at ZB authenticate against the Zimmer NAM AD
|
||||
([`ldap_setup.md`](../../ldap_setup.md)), which has two domain controllers
|
||||
(`uswap-vdc-nam1` / `uswap-vdc-nam2`). `ZB.MOM.WW.Auth.Ldap` takes a single
|
||||
`Server`, so a primary-DC outage blocks every interactive login family-wide
|
||||
(API keys unaffected). The working Ignition gateway has primary + secondary DC
|
||||
failover; this brings the shared library to parity on that axis.
|
||||
|
||||
## Decisions (made with the user, 2026-08-10)
|
||||
|
||||
1. **Sticky preference** failover — try the last-known-good DC first; on failure
|
||||
walk the ordered list and remember whichever worked. Mirrors the Secrets
|
||||
0.5.0 `FallbackEndpoints` sticky/no-flapping precedent. Rejected:
|
||||
always-primary-first (pays the full connection timeout on every login during
|
||||
a primary outage), sticky + probe-back (YAGNI for two DCs).
|
||||
2. **Failover-only** — no Ignition-style credential cache. A both-DCs-down event
|
||||
is rare and correlated with wider outages; API keys already survive it; a
|
||||
credential cache adds credential-at-rest surface. Possible future follow-up.
|
||||
3. **Approach A: service-level loop** — the walk lives in `LdapAuthService`,
|
||||
which already owns failure classification (user-fault vs system-fault).
|
||||
Rejected: connection-factory-level (only sees `Connect`; a DC that accepts
|
||||
TCP+TLS but fails service-bind/search would never fail over), DNS-level
|
||||
`nam.zimmer.com` (DC certs' SANs carry only their own FQDN → LDAPS hostname
|
||||
validation breaks; no stickiness or timeout control).
|
||||
|
||||
## Probe evidence (2026-08-10, recorded in `ldap_setup.md`)
|
||||
|
||||
Both DCs serve LDAPS (636, TLSv1.3) and StartTLS (389); anonymous rootDSE over
|
||||
LDAPS returns `result: 0 Success` on both. Certs: per-host FQDN SANs, issued by
|
||||
`zimmer-issueca1` ← `zimmer-rootca` (internal CA, root not served), valid to
|
||||
2027-03. **Caveat: both certs are RSA 1024-bit** — OpenSSL at SECLEVEL=2
|
||||
(Debian-based .NET containers) rejects the handshake (`EE certificate key too
|
||||
weak`); Windows/Schannel hosts accept. Container deployments need a lowered
|
||||
SECLEVEL, a validation-callback path, or (right fix) a reissued 2048-bit DC
|
||||
cert template.
|
||||
|
||||
## Config surface
|
||||
|
||||
One additive field on `LdapOptions` (`ZB.MOM.WW.Auth.Abstractions`):
|
||||
|
||||
```csharp
|
||||
/// <summary>Ordered fallback LDAP endpoints, "host" or "host:port"; a bare host uses <see cref="Port"/>.</summary>
|
||||
public IReadOnlyList<string> FallbackServers { get; init; } = Array.Empty<string>();
|
||||
```
|
||||
|
||||
```jsonc
|
||||
"Ldap": {
|
||||
"Server": "uswap-vdc-nam1.nam.zimmer.com",
|
||||
"Port": 636,
|
||||
"Transport": "Ldaps",
|
||||
"FallbackServers": [ "uswap-vdc-nam2.nam.zimmer.com" ]
|
||||
}
|
||||
```
|
||||
|
||||
- Empty (the default) ⇒ behavior byte-identical to today: single endpoint, no
|
||||
loop, no new state.
|
||||
- `ServerCertificateValidationCallback` and `AllowInsecure` apply to every
|
||||
endpoint (one CA family — verified by the probe).
|
||||
- **Validator** (`LdapOptionsValidator`, only when `Enabled`): each entry must
|
||||
have a non-empty host; an explicit port must parse as 1–65535. Malformed
|
||||
entries fail boot with a field-naming message in the validator's existing
|
||||
style. Duplicates are permitted (harmless; de-duplication is not the
|
||||
validator's job).
|
||||
|
||||
## Failover semantics
|
||||
|
||||
Inside `LdapAuthService.Authenticate`, an endpoint walk over an internal
|
||||
`LdapEndpoint(string Host, int Port)` list = `Server:Port` + parsed
|
||||
`FallbackServers`, in declared order:
|
||||
|
||||
- **Walk order:** sticky-preferred endpoint first, then the remaining endpoints
|
||||
in declared order (skipping the preferred). On success at endpoint *i*,
|
||||
preference := *i*.
|
||||
- **Only system-side failures advance the walk:** connect/TLS-upgrade failure,
|
||||
service-account bind failure, search failure, and any unexpected exception
|
||||
inside an attempt. **User-side results return immediately, never fail over:**
|
||||
`UserNotFound`, `AmbiguousUser`, `BadCredentials` (user bind),
|
||||
`GroupLookupFailed` — those are answers from a healthy directory.
|
||||
- **Exhaustion:** all endpoints failed system-side ⇒ return the last failure
|
||||
(maps to `ServiceAccountBindFailed`, as today — no `LdapAuthFailure` enum
|
||||
change, no `Abstractions` contract change beyond the new option).
|
||||
- **Fail-closed contract preserved:** `AuthenticateAsync` still never throws;
|
||||
each per-endpoint attempt is individually guarded.
|
||||
- Documented trade-offs:
|
||||
- A genuinely wrong service-account password tries every DC per login (fails
|
||||
fast at bind — no timeout cost; 2 endpoints in practice).
|
||||
- Worst-case login latency while the preferred DC is down =
|
||||
`ConnectionTimeoutMs` × walk position, paid once — stickiness then moves
|
||||
preference to the survivor.
|
||||
- Sticky means the primary stays unused after recovery until the current
|
||||
preferred fails; when it does, the walk tries declared order again, so
|
||||
preference drifts back to the primary naturally.
|
||||
|
||||
## Stickiness state
|
||||
|
||||
`volatile int _preferredIndex` on the singleton `LdapAuthService` (registered
|
||||
`TryAddSingleton` in `ZB.MOM.WW.Auth.AspNetCore`). Benign races tolerated —
|
||||
worst case is one redundant failed attempt. No cross-process or persisted
|
||||
state.
|
||||
|
||||
## Observability
|
||||
|
||||
New optional `ILogger<LdapAuthService>?` constructor parameter (null default —
|
||||
existing constructors keep compiling; adds only
|
||||
`Microsoft.Extensions.Logging.Abstractions` to `ZB.MOM.WW.Auth.Ldap`). The
|
||||
AspNetCore registration passes the host logger. Log points:
|
||||
|
||||
- **Warning** per failover hop: endpoint, stage that failed, next endpoint.
|
||||
- **Information** on sticky-preference change (old → new endpoint).
|
||||
- Nothing on the happy path; never log credentials or passwords.
|
||||
|
||||
Rationale: silent failover hides a dead primary DC indefinitely — the feature
|
||||
working correctly must be visible to operators.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit (FakeLdapConnection seam):** failover triggered by each system-side
|
||||
stage (connect, service bind, search, unexpected exception); NO failover on
|
||||
each user-side result; stickiness updates on success and drifts back after
|
||||
the preferred fails; exhaustion returns `ServiceAccountBindFailed`; empty
|
||||
`FallbackServers` short-circuits to today's exact behavior; endpoint parsing
|
||||
(`host`, `host:port`, malformed).
|
||||
- **Validator tests:** new cases in `LdapOptionsValidatorTests` style.
|
||||
- **Integration (env-gated GLAuth suite):** primary pointed at a closed local
|
||||
port + fallback at the real GLAuth ⇒ login succeeds, preference sticks, and a
|
||||
second login does not retry the dead primary. Fast offline too — a closed
|
||||
local port refuses immediately.
|
||||
|
||||
## Versioning / rollout
|
||||
|
||||
- Auth family `0.1.5` → **`0.2.0`** (additive feature; the four packages
|
||||
version together). Publish all 4 to the Gitea feed, restore-verify.
|
||||
- **Zero consumer code changes** — the four apps adopt by config alone at ZB
|
||||
deploy time. Bumping the apps' pins to 0.2.0 is a follow-up family-alignment
|
||||
task, not a prerequisite.
|
||||
- Dev/GLAuth stacks are unaffected (no `FallbackServers` configured).
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
# Zimmer NAM — LDAP / Active Directory Auth Setup
|
||||
|
||||
Reference for configuring services to authenticate against the Zimmer NAM domain.
|
||||
Extracted from the production Ignition 8.1 gateway `Ignition-IGNTN-APP-VP01`
|
||||
(`config.idb`, `NAM` AD-Hybrid user source), audited 2026-08-05. Full audit:
|
||||
`~/Desktop/config-idb-users-groups.md`.
|
||||
|
||||
## Directory / connection
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Domain | `nam.zimmer.com` |
|
||||
| Primary DC | `uswap-vdc-nam1:389` |
|
||||
| Secondary DC | `uswap-vdc-nam2:389` |
|
||||
| Encryption | **None** — plain LDAP on 389 (no LDAPS/StartTLS in the Ignition config) |
|
||||
| SASL | `DIGEST-MD5`, `CRAM-MD5` offered; QoP `auth-conf,auth-int,auth`; strength `high,medium,low`; no mutual auth |
|
||||
| Referrals | FOLLOW |
|
||||
| Read timeout | 60 000 ms |
|
||||
| Paging | 1000 entries/page |
|
||||
| Anonymous bind | Disallowed |
|
||||
|
||||
## Bind identity (UPN, not DN)
|
||||
|
||||
- Service account: `svc_igntnapp`
|
||||
- Binds by **UPN**: `svc_igntnapp@nam.zimmer.com` — there is no bind DN/CN anywhere
|
||||
in the working config; end users also authenticate as `<sAMAccountName>@nam.zimmer.com`
|
||||
(automatic suffix).
|
||||
- Password: held encrypted by the gateway; obtain from the credential owner — never
|
||||
commit it (in this repo's services use a `${secret:NAME}` reference).
|
||||
|
||||
## User lookup
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Search base | domain root `DC=nam,DC=zimmer,DC=com` (Ignition field left empty) |
|
||||
| User search filter | `(&(objectClass=user)(sAMAccountName={0}))` |
|
||||
| User enumeration filter | `(&(objectClass=user)(!(objectClass=computer)))` |
|
||||
| Username attribute | `sAMAccountName` |
|
||||
| Display name | `name` |
|
||||
| Email | `mail` |
|
||||
| Phone / SMS | `telephoneNumber` / `mobile` |
|
||||
|
||||
## Groups / roles
|
||||
|
||||
**The working setup does not use LDAP groups at all.** The Ignition profile is
|
||||
AD-Hybrid: AD answers *who you are*, while *what you may do* (roles
|
||||
Administrator, Designer, Engineer, Maintenance, Management, Mfg_Tech, Operator)
|
||||
is assigned manually per user inside the application. AD security-group
|
||||
membership grants nothing.
|
||||
|
||||
Implication for new services: don't assume a `memberOf`-based authorization
|
||||
scheme exists to mirror — there is no established group→role convention in this
|
||||
domain's SCADA stack. Either keep app-local role assignment (matching the
|
||||
existing pattern) or define new AD groups deliberately; standard AD attributes
|
||||
(`memberOf`, filter `(&(objectClass=group)(member={0}))`) are available if we
|
||||
choose to.
|
||||
|
||||
## Behavioral notes from the working config
|
||||
|
||||
- Users are **populated on demand**: a local record is created on first
|
||||
successful AD login (with no roles until granted).
|
||||
- Failover: soft failover to the local internal user source, 60 s credential-cache
|
||||
validation — cached logins keep working through a DC outage.
|
||||
- Lockout: 5 failed attempts / 15-minute window.
|
||||
|
||||
## Cautions
|
||||
|
||||
- Port 389 without TLS means simple binds transit cleartext unless SASL
|
||||
confidentiality is negotiated — for new .NET services prefer LDAPS (636) or
|
||||
StartTLS against the same DCs.
|
||||
|
||||
## TLS probe results (verified 2026-08-10, from the dev Mac)
|
||||
|
||||
Both DCs were live-probed; **LDAPS and StartTLS both work on both DCs.**
|
||||
|
||||
| Check | uswap-vdc-nam1 | uswap-vdc-nam2 |
|
||||
|---|---|---|
|
||||
| Ports open | 389, 636, 3268, 3269 | 389, 636, 3268, 3269 |
|
||||
| LDAPS (636) handshake | TLSv1.3 OK | TLSv1.3 OK |
|
||||
| StartTLS on 389 | OK | OK |
|
||||
| Anonymous rootDSE over LDAPS | `result: 0 Success` | `result: 0 Success` |
|
||||
| Cert subject/SAN | `USWAP-VDC-NAM1.nam.zimmer.com` (matches FQDN) | `USWAP-VDC-NAM2.nam.zimmer.com` (matches FQDN) |
|
||||
| Cert validity | 2026-03-21 → 2027-03-21 | 2026-03-20 → 2027-03-20 |
|
||||
| Chain | leaf ← `zimmer-issueca1` ← `zimmer-rootca` (intermediate served, root not) | same |
|
||||
| **Key size** | **RSA 1024-bit** (sha256 sig) | **RSA 1024-bit** (sha256 sig) |
|
||||
|
||||
Implications for new .NET services:
|
||||
|
||||
- **Use LDAPS 636** (or StartTLS) — no reason to accept cleartext binds.
|
||||
- **Trust:** `zimmer-rootca` is an internal CA, not in default OS/container trust
|
||||
stores. Either install the root (+ intermediate) into the host/container trust
|
||||
store, or pin the chain via `LdapOptions.ServerCertificateValidationCallback`
|
||||
(code-only seam in `ZB.MOM.WW.Auth.Ldap`).
|
||||
- **⚠️ RSA 1024-bit DC certs:** OpenSSL at default security level (SECLEVEL=2 —
|
||||
what Debian-based .NET containers ship) **rejects 1024-bit keys**
|
||||
(`EE certificate key too weak`, observed in this probe), so a Linux/container
|
||||
client may fail the TLS handshake even with the CA trusted. Windows/Schannel
|
||||
hosts accept them. Options: get the DC cert template reissued at 2048+ (right
|
||||
fix, needs AD PKI owner), or lower SECLEVEL / use a custom validation callback
|
||||
in affected containers (workaround). Certs expire 2027-03 — a reissue
|
||||
opportunity.
|
||||
- Hostnames resolve from the dev network: nam1 = `10.201.216.2`,
|
||||
nam2 = `10.201.138.78`.
|
||||
|
||||
Reference in New Issue
Block a user