docs(ldap): NAM AD setup notes + live TLS probe results; backup-DC failover design (Auth 0.2.0)
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,139 @@
|
||||
# LDAP backup-DC failover — design (`ZB.MOM.WW.Auth` 0.2.0)
|
||||
|
||||
**Date:** 2026-08-10
|
||||
**Status:** Approved
|
||||
**Motivation:** Production servers at ZB authenticate against the Zimmer NAM AD
|
||||
([`ldap_setup.md`](../../ldap_setup.md)), which has two domain controllers
|
||||
(`uswap-vdc-nam1` / `uswap-vdc-nam2`). `ZB.MOM.WW.Auth.Ldap` takes a single
|
||||
`Server`, so a primary-DC outage blocks every interactive login family-wide
|
||||
(API keys unaffected). The working Ignition gateway has primary + secondary DC
|
||||
failover; this brings the shared library to parity on that axis.
|
||||
|
||||
## Decisions (made with the user, 2026-08-10)
|
||||
|
||||
1. **Sticky preference** failover — try the last-known-good DC first; on failure
|
||||
walk the ordered list and remember whichever worked. Mirrors the Secrets
|
||||
0.5.0 `FallbackEndpoints` sticky/no-flapping precedent. Rejected:
|
||||
always-primary-first (pays the full connection timeout on every login during
|
||||
a primary outage), sticky + probe-back (YAGNI for two DCs).
|
||||
2. **Failover-only** — no Ignition-style credential cache. A both-DCs-down event
|
||||
is rare and correlated with wider outages; API keys already survive it; a
|
||||
credential cache adds credential-at-rest surface. Possible future follow-up.
|
||||
3. **Approach A: service-level loop** — the walk lives in `LdapAuthService`,
|
||||
which already owns failure classification (user-fault vs system-fault).
|
||||
Rejected: connection-factory-level (only sees `Connect`; a DC that accepts
|
||||
TCP+TLS but fails service-bind/search would never fail over), DNS-level
|
||||
`nam.zimmer.com` (DC certs' SANs carry only their own FQDN → LDAPS hostname
|
||||
validation breaks; no stickiness or timeout control).
|
||||
|
||||
## Probe evidence (2026-08-10, recorded in `ldap_setup.md`)
|
||||
|
||||
Both DCs serve LDAPS (636, TLSv1.3) and StartTLS (389); anonymous rootDSE over
|
||||
LDAPS returns `result: 0 Success` on both. Certs: per-host FQDN SANs, issued by
|
||||
`zimmer-issueca1` ← `zimmer-rootca` (internal CA, root not served), valid to
|
||||
2027-03. **Caveat: both certs are RSA 1024-bit** — OpenSSL at SECLEVEL=2
|
||||
(Debian-based .NET containers) rejects the handshake (`EE certificate key too
|
||||
weak`); Windows/Schannel hosts accept. Container deployments need a lowered
|
||||
SECLEVEL, a validation-callback path, or (right fix) a reissued 2048-bit DC
|
||||
cert template.
|
||||
|
||||
## Config surface
|
||||
|
||||
One additive field on `LdapOptions` (`ZB.MOM.WW.Auth.Abstractions`):
|
||||
|
||||
```csharp
|
||||
/// <summary>Ordered fallback LDAP endpoints, "host" or "host:port"; a bare host uses <see cref="Port"/>.</summary>
|
||||
public IReadOnlyList<string> FallbackServers { get; init; } = Array.Empty<string>();
|
||||
```
|
||||
|
||||
```jsonc
|
||||
"Ldap": {
|
||||
"Server": "uswap-vdc-nam1.nam.zimmer.com",
|
||||
"Port": 636,
|
||||
"Transport": "Ldaps",
|
||||
"FallbackServers": [ "uswap-vdc-nam2.nam.zimmer.com" ]
|
||||
}
|
||||
```
|
||||
|
||||
- Empty (the default) ⇒ behavior byte-identical to today: single endpoint, no
|
||||
loop, no new state.
|
||||
- `ServerCertificateValidationCallback` and `AllowInsecure` apply to every
|
||||
endpoint (one CA family — verified by the probe).
|
||||
- **Validator** (`LdapOptionsValidator`, only when `Enabled`): each entry must
|
||||
have a non-empty host; an explicit port must parse as 1–65535. Malformed
|
||||
entries fail boot with a field-naming message in the validator's existing
|
||||
style. Duplicates are permitted (harmless; de-duplication is not the
|
||||
validator's job).
|
||||
|
||||
## Failover semantics
|
||||
|
||||
Inside `LdapAuthService.Authenticate`, an endpoint walk over an internal
|
||||
`LdapEndpoint(string Host, int Port)` list = `Server:Port` + parsed
|
||||
`FallbackServers`, in declared order:
|
||||
|
||||
- **Walk order:** sticky-preferred endpoint first, then the remaining endpoints
|
||||
in declared order (skipping the preferred). On success at endpoint *i*,
|
||||
preference := *i*.
|
||||
- **Only system-side failures advance the walk:** connect/TLS-upgrade failure,
|
||||
service-account bind failure, search failure, and any unexpected exception
|
||||
inside an attempt. **User-side results return immediately, never fail over:**
|
||||
`UserNotFound`, `AmbiguousUser`, `BadCredentials` (user bind),
|
||||
`GroupLookupFailed` — those are answers from a healthy directory.
|
||||
- **Exhaustion:** all endpoints failed system-side ⇒ return the last failure
|
||||
(maps to `ServiceAccountBindFailed`, as today — no `LdapAuthFailure` enum
|
||||
change, no `Abstractions` contract change beyond the new option).
|
||||
- **Fail-closed contract preserved:** `AuthenticateAsync` still never throws;
|
||||
each per-endpoint attempt is individually guarded.
|
||||
- Documented trade-offs:
|
||||
- A genuinely wrong service-account password tries every DC per login (fails
|
||||
fast at bind — no timeout cost; 2 endpoints in practice).
|
||||
- Worst-case login latency while the preferred DC is down =
|
||||
`ConnectionTimeoutMs` × walk position, paid once — stickiness then moves
|
||||
preference to the survivor.
|
||||
- Sticky means the primary stays unused after recovery until the current
|
||||
preferred fails; when it does, the walk tries declared order again, so
|
||||
preference drifts back to the primary naturally.
|
||||
|
||||
## Stickiness state
|
||||
|
||||
`volatile int _preferredIndex` on the singleton `LdapAuthService` (registered
|
||||
`TryAddSingleton` in `ZB.MOM.WW.Auth.AspNetCore`). Benign races tolerated —
|
||||
worst case is one redundant failed attempt. No cross-process or persisted
|
||||
state.
|
||||
|
||||
## Observability
|
||||
|
||||
New optional `ILogger<LdapAuthService>?` constructor parameter (null default —
|
||||
existing constructors keep compiling; adds only
|
||||
`Microsoft.Extensions.Logging.Abstractions` to `ZB.MOM.WW.Auth.Ldap`). The
|
||||
AspNetCore registration passes the host logger. Log points:
|
||||
|
||||
- **Warning** per failover hop: endpoint, stage that failed, next endpoint.
|
||||
- **Information** on sticky-preference change (old → new endpoint).
|
||||
- Nothing on the happy path; never log credentials or passwords.
|
||||
|
||||
Rationale: silent failover hides a dead primary DC indefinitely — the feature
|
||||
working correctly must be visible to operators.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit (FakeLdapConnection seam):** failover triggered by each system-side
|
||||
stage (connect, service bind, search, unexpected exception); NO failover on
|
||||
each user-side result; stickiness updates on success and drifts back after
|
||||
the preferred fails; exhaustion returns `ServiceAccountBindFailed`; empty
|
||||
`FallbackServers` short-circuits to today's exact behavior; endpoint parsing
|
||||
(`host`, `host:port`, malformed).
|
||||
- **Validator tests:** new cases in `LdapOptionsValidatorTests` style.
|
||||
- **Integration (env-gated GLAuth suite):** primary pointed at a closed local
|
||||
port + fallback at the real GLAuth ⇒ login succeeds, preference sticks, and a
|
||||
second login does not retry the dead primary. Fast offline too — a closed
|
||||
local port refuses immediately.
|
||||
|
||||
## Versioning / rollout
|
||||
|
||||
- Auth family `0.1.5` → **`0.2.0`** (additive feature; the four packages
|
||||
version together). Publish all 4 to the Gitea feed, restore-verify.
|
||||
- **Zero consumer code changes** — the four apps adopt by config alone at ZB
|
||||
deploy time. Bumping the apps' pins to 0.2.0 is a follow-up family-alignment
|
||||
task, not a prerequisite.
|
||||
- Dev/GLAuth stacks are unaffected (no `FallbackServers` configured).
|
||||
Reference in New Issue
Block a user