docs(former-api-specs): legacy MES alarm API — code-verified behaviour notes

Checked against the legacy mesapi source (MesNotifier/MesServices/Tag) plus ServiceStack/OrmLite
6.0.2 in a scratch harness:
- unhandled exceptions are an HTTP 500 ServiceStack ErrorResponse, not WasSuccessful=false
- no matching alarms = success with an empty list; failures don't short-circuit
- AckDT is null only for a null MXAccess value, not 'when unacked'
- tag type-conversion failures are silent (InAlarm false -> alarm drops out)
- simplealarmstatus SAPID edge cases: null never matches, "" matches, duplicates -> arbitrary TOP 1
- 'test' API keys hit the production BatchDB (TestDb routing never reaches MesNotifier)
- latent SetResult-after-timeout race in the MXAccess data-change callback

docs/former-api-specs/mes/Alarm-API.md

@@ -46,8 +46,9 @@ The caller must be authenticated **and** hold the `MESAPI` role. Auth is configu
 - **API key** — `ApiKeyAuthProvider`
   - `SessionCacheDuration = 30 minutes`
   - `RequireSecureConnection = false` (HTTP is accepted; TLS not enforced)
-  - An API key whose `Environment == "test"` is routed to the `TestDb` connection instead of the
+  - `AppHost.GetDbConnection` routes an API key whose `Environment == "test"` to a `TestDb`
-    production DB (`AppHost.GetDbConnection`).
+    connection — **but this does NOT apply to these endpoints** (see *Behavior notes*: test keys
+    query the production `BatchDB`).
 - **LDAP** — `LdapAuthProvider` (directory credentials).
 - `AllowGetAuthenticateRequests = true`.
 
@@ -160,7 +161,7 @@ fails with a selector-specific message, e.g. `"Failed to find machine with Code
 | `Severity` | int | `MachineAlarm.Severity` (0–999). |
 | `StatusCode` | string | `"Triggered"`, or `"Triggered.Acked"` when acked. |
 | `TriggeredDT` | DateTime | Live `…​.TimeAlarmOn` tag value. |
-| `AckDT` | DateTime? | Live `…​.TimeAlarmAcked` tag value (null if unacked). |
+| `AckDT` | DateTime? | Live `…​.TimeAlarmAcked` tag value — read for every triggered alarm, acked or not; null only if MXAccess delivers a null value (see *Behavior notes*). |
 | `AckComment` | string | Live `…​.AckMsg` tag value. |
 
 ### Example response
@@ -237,6 +238,50 @@ SQL table backing alarm configuration (`APIServer.ServiceModel/DTO/MachineAlarm.
 - **All-or-nothing on read errors.** A single bad-quality tag fails the whole response (success
   flag flips, alarm list is emptied) rather than returning a partial set.
 - **Severity bounds are inclusive** on both `MinSeverity` and `MaxSeverity`.
+- **Unhandled exceptions are an HTTP 500, not `WasSuccessful = false`.** Neither `MesServices`
+  handler catches anything (they block on `.Result`) and `AppHost` registers no exception handlers.
+  A SQL error, an exception from the MXAccess COM calls (`AddItem` / `AdviseSupervisory`) or
+  cancellation while opening the DB connection returns ServiceStack's error body (response DTO has
+  no `ResponseStatus`, so the generic `ErrorResponse` is used; `ErrorCode` = the *inner* exception
+  type name, `Message` = its raw text, `StackTrace` null outside Debug builds):
+  ```
+  HTTP 500
+  {"ResponseStatus":{"ErrorCode":"SqlException","Message":"<raw exception text>","StackTrace":null,"Errors":[]}}
+  ```
+  A caller without `Accept: application/json` gets ServiceStack's HTML error page instead. The
+  error is also written to `requestlogs/{year}-{month}/{…}-errors.csv`. (Behaviour reproduced on
+  ServiceStack 6.0.2 with the same handler shape, 2026-09-28.) The 30-second read timeout does
+  **not** throw — pending reads resolve as failed and yield the normal `WasSuccessful = false` body.
+- **No matching alarms = success with an empty list.** A machine with no `MachineAlarm` rows (or,
+  on `/simplealarmstatus`, no `FlaggedForMES` rows) returns `WasSuccessful = true`, `Alarms: []` —
+  indistinguishable from "configured alarms, none in alarm".
+- **Failures don't short-circuit.** After a quality/read failure the handler still reads the detail
+  tags and builds the list, then clears it before returning — cost only (time + MXAccess load).
+- **`AckDT` is not guaranteed null for an unacked alarm.** `Tag<T>.ParseUpdate` yields null only
+  for a null MXAccess value; otherwise it is whatever `TimeAlarmAcked` holds. A Galaxy time
+  attribute normally holds a date, so expect a stale/default timestamp rather than null
+  (unverified against a live Galaxy). Use `StatusCode` to decide acked vs. unacked.
+- **Tag type-conversion failures are silent.** If a value fails `Convert.ChangeType`,
+  `ParseUpdate` sets the tag to `default(T)` and `IsGood = false`, but the read still reports success
+  (it keys on MXAccess quality only) and only the `Quality` tag's `IsGood` is checked. So a bad
+  `InAlarm` reads as `false` and the alarm silently drops out; a bad `TimeAlarmOn` becomes
+  `0001-01-01T00:00:00`; a bad `DescAttrName`/`AckMsg` becomes null.
+- **`/simplealarmstatus` machine lookup edge cases** (SQL is `SELECT TOP 1 … WHERE SAPID = @0`,
+  confirmed on OrmLite 6.0.2; the column is nullable and not unique):
+  - missing / null `SAPID` → parameter is `NULL`, `= NULL` never matches, so machines with a NULL
+    SAPID are **not** returned → `"Failed to find machine with SAPID ''"`;
+  - `"SAPID": ""` **does** match a machine whose SAPID is the empty string (the full endpoint
+    guards with `IsNullOrEmpty` and never issues this query);
+  - duplicate SAPIDs → `TOP 1` with no `ORDER BY`, so SQL Server picks an arbitrary machine. The
+    same applies to `SAPID`/`Code`/`ZTag` lookups on `/mes/alarmstatus`.
+- **"test" API keys hit the production database.** `AppHost.GetDbConnection`'s `TestDb` routing
+  only affects a service's `Db` property; `MesNotifier` opens its own connection from the
+  registered `OrmLiteConnectionFactory` (`MesNotifier.cs` lines 71/168/347/525), which is always
+  `BatchDB`. No `TestDb` named connection is registered either. Applies to MoveIn/MoveOut too.
+- **Late-arriving value after timeout (latent).** `LmxProxyOnOnDataChange` completes a pending read
+  with `SetResult`, not `TrySetResult`; a value that lands after the 30 s timeout already resolved
+  the read (before its `ContinueWith` removes it) would throw inside the MXAccess COM callback.
+  Not reproduced.
 - The route attributes carry placeholder Swagger text (`Summary = "POST Summary"`,
   `Notes = "Notes"`); these are cosmetic.