ScadaBridge/docs/api-specs/Delmia-Recipe-Download-API.md

DelmiaRecipeDownload — ScadaBridge Inbound API spec (as deployed)

Captured: 2026-09-02 from the live prod management API on wonder-app-vd03 (method id 4, API keys, the CvdReactor template's ProcessRecipeDownload script, the DelmiaReceiver template, and the inbound audit log). QA (vq03/vq04) was NOT read — its management API requires real LDAP credentials (prod runs with DisableLogin); re-capture QA with scadabridge api-method get --id … once credentials are to hand. The definition is config-as-data in the central ConfigDb (not in git), so this file is the durable record; the legacy predecessor is documented in ../former-api-specs/dnc/Delmia-Integration-API.md (Surface B), and the client that calls it is the WWNotifier.exe drop-in (DelmiaNotifier).

1. Endpoint

Item	Value
Method / path	POST /api/DelmiaRecipeDownload
Auth	X-API-Key: sbk_<keyId>_<secret> or Authorization: Bearer sbk_<keyId>_<secret> (Authorization wins if both)
Content type	application/json, flat PascalCase object
Server timeout	45 s (TimeoutSeconds on the method); the site-side handshake wait inside it is 25 s
Prod base URL	http://wonder-app-vd03.zmr.zimmer.com:8085 (single central node)
QA base URLs	http://10.220.158.2:8085 (vq03), http://10.220.158.3:8085 (vq04) — list both in the notifier's BaseUrls; a standby central relays to the active peer

Unauthenticated / bad-key calls return 401 {"error":"Invalid or missing API key"} (audit kind InboundAuthFailure). General Inbound API status semantics (413 body cap, 503 STANDBY_NODE / ACTIVE_NODE_UNREACHABLE, unknown method) are in Component-InboundAPI.md.

2. Request

Deployed ParameterDefinitions (JSON Schema, verbatim — note required is empty; the script itself rejects a missing/blank MachineCode):

{"type":"object","properties":{
  "MachineCode":{"type":"string"},
  "DownloadPath":{"type":"string"},
  "WorkOrderNumber":{"type":"string"},
  "PartNumber":{"type":"string"},
  "JobStepNumber":{"type":"string"},
  "Username":{"type":"string"}},
 "required":[]}

Field	Notifier flag	Meaning	Where it lands on the site
MachineCode	-m	Routing key — must equal an instance UniqueName exactly (trimmed). Prod today: Z28061, Z28062, Z28061Sim.	Route.To(MachineCode)
DownloadPath	-d	Path where the recipe/NC file was placed	DelmiaReceiver.DownloadPath
WorkOrderNumber	-w	Work order	DelmiaReceiver.WorkOrderNumber
PartNumber	-p	Part / item number	DelmiaReceiver.PartNumber
JobStepNumber	-s	Job step / sequence op	DelmiaReceiver.JobStepNumber
Username	-u	Operator (audit only)	DelmiaReceiver.Username

Example (a real call taken from the prod audit log):

curl -s -X POST http://wonder-app-vd03.zmr.zimmer.com:8085/api/DelmiaRecipeDownload \
  -H "X-API-Key: $SCADABRIDGE_API_KEY" -H "Content-Type: application/json" \
  -d '{"MachineCode":"Z28061Sim","DownloadPath":"C:\\recipes\\test.nc","WorkOrderNumber":"WO-TEST-001","PartNumber":"PN-TEST","JobStepNumber":"0100","Username":"delmia-sim-test"}'

3. Response

Deployed ReturnDefinition: {"type":"object","properties":{"Result":{"type":"boolean"},"ResultText":{"type":"string"}},"required":["Result","ResultText"]}

HTTP 200 is returned for every business outcome — success is Result == true, never the status code. ResultText values the deployed scripts can produce:

Result	ResultText	Produced by
true	echo of DelmiaReceiver.RecipeProcessResultText (often "")	Galaxy object acknowledged the download
false	DelmiaReceiver.RecipeProcessResultText	Galaxy object rejected the download (RecipeProcessResult false)
false	MachineCode is required	central script, blank MachineCode
false	Receiver ready flag not set to true	site script, ReadyFlag not true (or unreadable — see §7)
false	Timeout waiting for recipe to be processed	site script, RecipeProcessedFlag not true within 25 s
false	RecipeDownload failed: <exception>	either script — e.g. unknown instance, site unreachable, Script 'ProcessRecipeDownload' on instance 'X' failed: …

4. Processing flow

Delmia ─► WWNotifier.exe ─► POST /api/DelmiaRecipeDownload (central, active node)
                              │ inbound method script (id 4): validate MachineCode
                              ▼
                     Route.To(MachineCode).Call("ProcessRecipeDownload", {5 fields})
                              │ gRPC central→site, lands on the instance actor
                              ▼
      CvdReactor.ProcessRecipeDownload (site Call script) on the composed DelmiaReceiver child:
        1. read  ReadyFlag            → must be true, else "Receiver ready flag not set to true"
        2. write RecipeProcessedFlag  = false
        3. WriteBatchAndWaitAsync({DownloadPath, WorkOrderNumber, PartNumber, JobStepNumber, Username},
                                  trigger RecipeDownloadFlag = true,
                                  wait RecipeProcessedFlag == true, timeout 25 s)
        4. read  RecipeProcessResult / RecipeProcessResultText → {Result, ResultText}

Both scripts as deployed on prod (verbatim):

Central inbound method script (id 4, timeout 45 s)

try {
  System.Func<string, object> P = k => Parameters.ContainsKey(k) ? Parameters[k] : null;
  var code = (P("MachineCode") as string ?? "").Trim();
  if (string.IsNullOrEmpty(code)) return new { Result = false, ResultText = "MachineCode is required" };
  var suf = code.Substring(code.Length - 1);

  return await Route.To(code).Call("ProcessRecipeDownload", new {
      DownloadPath    = P("DownloadPath"),
      WorkOrderNumber = P("WorkOrderNumber"),
      PartNumber      = P("PartNumber"),
      JobStepNumber   = P("JobStepNumber"),
      Username        = P("Username"),
  });
} catch (Exception ex) {
  return new { Result = false, ResultText = "RecipeDownload failed: " + ex.Message };
}

Site script CvdReactor.ProcessRecipeDownload (template 1, script id 4, trigger call)

// Generic DELMIA recipe-download — performed DIRECTLY on the composed DelmiaReceiver child
// (mirrors how MesMoveIn writes the MESReceiver child) instead of delegating via CallScript.
// No side logic; the receiver's own DelmiaReceiver.Download method holds the same handshake.
var recv = Children["DelmiaRecv"];
System.Func<object,bool> IsTrue = v => { var s=v?.ToString(); return (v is bool b && b)||s=="1"||string.Equals(s,"true",System.StringComparison.OrdinalIgnoreCase); };
System.Func<string,object?> P = k => Parameters.ContainsKey(k) ? Parameters[k] : null;
if (!IsTrue(recv.Attributes["ReadyFlag"]))
    return new { Result=false, ResultText="Receiver ready flag not set to true" };
try {
    recv.Attributes["RecipeProcessedFlag"] = false;
    var data = new System.Collections.Generic.Dictionary<string, object?> {
        { "DownloadPath", P("DownloadPath") },
        { "WorkOrderNumber", P("WorkOrderNumber") },
        { "PartNumber", P("PartNumber") },
        { "JobStepNumber", P("JobStepNumber") },
        { "Username", P("Username") },
    };
    if (!await recv.Attributes.WriteBatchAndWaitAsync(data, "RecipeDownloadFlag", true, "RecipeProcessedFlag", true, System.TimeSpan.FromSeconds(25)))
        return new { Result=false, ResultText="Timeout waiting for recipe to be processed" };
    return new {
        Result = IsTrue(recv.Attributes["RecipeProcessResult"]),
        ResultText = recv.Attributes["RecipeProcessResultText"]?.ToString() ?? ""
    };
} catch (Exception ex) {
    return new { Result=false, ResultText="RecipeDownload failed: " + ex.Message };
}

Known race (Gitea #36, 2026-09-10). Step 2's RecipeProcessedFlag = false is a device write whose echo has not arrived when step 3's wait registers, so the wait can be satisfied by the PREVIOUS download's cached true. The runtime fix is the reset-aware overload — WriteBatchAndWaitAsync(data, "RecipeDownloadFlag", true, "RecipeProcessedFlag", true, 25 s, "RecipeProcessedFlag", false, 10 s) returning a HandshakeResult — which replaces steps 2–3. The deployed prod/QA script bodies are NOT yet ported (deferred-work register row 119).

5. Galaxy-side contract — the DelmiaReceiver template (id 12; composed clone id 26)

All ten attributes are data-sourced over the site MxGateway connection, bound per instance to the reactor-level Galaxy object (prod: DelmiaReceiver_038 on Z28061, _039 on Z28062, _037 on Z28061Sim; Left_00x.DelmiaReceiver / Right_00x.DelmiaReceiver back the unused side compositions).

Attribute	Type	Direction	Role in the handshake
ReadyFlag	Boolean	Galaxy → SB	Precondition; must be true or the call is refused
RecipeDownloadFlag	Boolean	SB → Galaxy	Trigger — set true after the five data fields are written
RecipeProcessedFlag	Boolean	both	SB clears to false first; Galaxy sets true when done (waited ≤ 25 s)
RecipeProcessResult	Boolean	Galaxy → SB	Outcome → Result
RecipeProcessResultText	String	Galaxy → SB	Outcome text → ResultText
DownloadPath, WorkOrderNumber, PartNumber, JobStepNumber, Username	String	SB → Galaxy	Payload, written as one batch before the trigger

6. API keys scoped to this method (prod, names only — secrets are shown once at creation)

mes, mes api key, delmia-mes-receivers, TestKey, JOE_TEST_KEY — all enabled, all also scoped to the MES methods (MesMoveIn, MesMoveOut, MesSampleData, SimpleAlarmStatusRequest, most also IpsenMESMoveIn). delmia-mes-receivers is the one whose name says it is the Delmia caller's key; the audit actor column shows which key served each call.

7. Observed behaviour and open issues (from the prod audit log, 2026-06-17 → 2026-06-26)

Nine calls total, all test traffic (actors delmia-mes-receivers, delmia-sim-test), none since 2026-06-26 — no production Delmia traffic has hit this method yet. Every 200 answered Result=false: mostly Receiver ready flag not set to true, one Script 'ProcessRecipeDownload' … The given key 'side' was not present from an earlier script revision, and one MachineCode must end with A (Left) or B (Right) from a retired left/right variant (the current script routes on the whole MachineCode; the leftover suf variable is unused).

1. Likely broken since the 2026-06-27 rename — verify before go-live. The site script still addresses the child as Children["DelmiaRecv"], but the CvdReactor composition was renamed DelmiaRecv → DelmiaReceiver that day (../deployment/deployment-records/2026-06-27-wonder-app-vd03.md §2b), so the flattened paths are DelmiaReceiver.*. ChildrenAccessor just concatenates the name into a path and GetAttribute on an unknown path logs a warning and yields null, so IsTrue(null) is false and every call will answer Receiver ready flag not set to true regardless of the real flag. No call has been made since the rename, so this is inferred from the code, not observed. Fix = change that one line to Children["DelmiaReceiver"] on the template (scadabridge template script update) and redeploy the three CvdReactor instances; check QA's copy of the script for the same string.
2. Not idempotent. The design requires idempotency on (MachineCode, DownloadPath, WorkOrderNumber) because the notifier retries on timeout/connect failure (at-least-once). The deployed script re-runs the full handshake on a duplicate, so a retry after a slow first attempt will trigger a second download on the Galaxy object.
3. required is empty in the parameter schema; only MachineCode is enforced, in script. Missing payload fields are written to the receiver as null/empty.
4. QA definitions unread (credentials) — assumed identical (same build lineage), not verified.

8. Docker rig stand-in and notifier verification (2026-09-02)

The local rig (docker/, central-a :9001 / central-b :9002 / Traefik :9000, site-a = site id 1) carries an in-memory stand-in for a vd03-style machine so the whole path can be exercised without a Galaxy. Seeded by CLI only, no DB edits:

• docker/seed-delmia-sim.sh — template DelmiaReceiverSim (the ten §5 attributes, all static, ReadyFlag defaulting to true) composed into SimMachine as DelmiaReceiver; instance DelmiaSim01 on site-a, deployed. SimMachine carries two scripts:
  • ProcessRecipeDownload (Call) — the §4 handshake ported to static attributes. WriteBatchAndWaitAsync rejects static attributes (InstanceActor.HandleWriteAttributeBatch: "not a data-sourced attribute"), so the port writes the payload with per-attribute SetAsync and waits with WaitAsync — same trigger/response flags, same 25 s budget, same result mapping. It addresses Children["DelmiaReceiver"] (the post-rename name; contrast §7 item 1).
  • DelmiaReceiverSimulator (ValueChange on DelmiaReceiver.RecipeDownloadFlag) — plays the Galaxy object: on the rising edge it accepts unless DownloadPath ends in .bad or the work order is blank, writes RecipeProcessResult/RecipeProcessResultText, clears the trigger flag, then sets RecipeProcessedFlag = true. It lives on the root template because composed scripts' trigger configs are not prefixed at flattening, and ScriptActor matches ValueChange by exact canonical name.
• docker/seed-delmia-api.sh — the §4 method verbatim (minus its dead suf line), timeout 45 s, plus one key (delmia-notifier) scoped to it; the token is printed once.
• docker/sync-inbound-api-keys.sh — required before any central failover test, see below. Superseded 2026-09-02 by register row 108 / the shared SQL Server key store (ScadaBridge:InboundApi:ApiKeyStore:Provider=SqlServer, now set on both rig centrals): the script is deleted and verify-delmia-notifier.sh needs no file copy before a failover run — both nodes read the same zbauth rows.
• docker/verify-delmia-notifier.sh — runs the managed WWNotifier.dll from a scratch copy whose appsettings.json carries the key (ScadaBridge:ApiKey), with BaseUrls listing the two central nodes directly (not Traefik) so the notifier's own connect-failure failover is what gets exercised.

Whole-path latency through Traefik: a curl call returns in under 100 ms (simulator fires on the same node, no device round-trip).

Verified outcomes (second run, 2026-09-02 16:51 UTC; row H from the third run, 2026-09-06 04:48 UTC — A–G re-passed, takeover 29 s, rejoin 3 s)

Case	Setup	Result	Audit row (ApiInbound)
A	active (central-a) first in BaseUrls	YES	InboundRequest on central-a
B	standby (central-b) first — standby relays	YES	one row, on central-a (the relay does not double-audit)
C	bad key	NO + HTTP 401	InboundAuthFailure on central-a
D	active docker killed, attempt inside the takeover window	NO + HTTP 503 (ACTIVE_NODE_UNREACHABLE from the not-yet-active survivor) — expected, connect-failure-only failover	none (infrastructure answer)
E	survivor /health/active = 200 — takeover measured 27 s after the kill	YES (:9001 refused → :9002 200)	InboundRequest on central-b
F	victim restarted, memberCount=2, victim /health/active = 503 (rejoined as standby)	YES (victim relays)	InboundRequest on central-b
G	DownloadPath = C:\recipes\x.bad	NO + Simulated reject: bad path or missing work order	InboundRequest on central-b, Result=false
H	Flow Debugger (2026-09-06 third run, branch feat/flow-debugger): flow-debug run-method on DelmiaRecipeDownload, then flow-debug replay on the returned root id	YES — live run completed, 11 events inline (InboundRequest, RouteCall, ScriptRun, 7 × AttributeWrite, AttributeWait); replay rebuilt the same root with 17 events on the first attempt (no telemetry lag hit)	ApiInbound/InboundRequest (actor = operator) + ScriptTrace rows; the six extra replay rows are the simulator's two cascaded ValueChange runs (rising + clearing edge of RecipeDownloadFlag, 2 × ScriptRun + 4 × AttributeWrite) — inline TraceEvents carry only the routed run's own steps, replay walks the whole execution tree

The notifier's first-line contract, exit codes (0 / -1), and the reason line on NO behaved as the README states throughout.

Two findings from the first run (kept, not "fixed")

1. docker stop is not a crash. SIGTERM triggers CoordinatedShutdown's graceful singleton handover — the survivor was active 0 s after the stop. The drill uses docker kill so auto-down's ~25–30 s takeover is what gets measured (docker/failover-drill.sh already does the same).

2. Inbound API keys are a per-node SQLite store, so a central failover invalidates every client's key. Keys live in ScadaBridge:InboundApi:ApiKeyStore:SqlitePath (/app/data/inbound-api-keys.sqlite, on each central's own data volume), and key admin (ManagementActor, a per-node ActorOf reached from that node's /management endpoints) writes only the node that serves the call — through Traefik that is the active node. After the first run's takeover, central-b answered 401 KeyNotFound to the key central-a had minted; its security api-key list was []. The standby-relay case (B) passes only because the standby forwards before authenticating. The same asymmetry means a key revoked on one node stays valid on the other. Rig workaround at the time: docker/sync-inbound-api-keys.sh checkpointed central-a's store on the host (sqlite3) and replaced central-b's (target stopped for the copy; both rig nodes share one pepper, so the hashes verify on either node); the verify script's preflight refused to start unless both nodes listed the key.

   Superseded 2026-09-02 (same day) by register row 108. ZB.MOM.WW.Auth 0.3.0 adds a SQL Server provider (ZB.MOM.WW.Auth.ApiKeys.SqlServer); both rig centrals now run ScadaBridge:InboundApi:ApiKeyStore:Provider=SqlServer, sharing one zbauth row set in the configuration database, so a key exists on both nodes the instant it's created and a revoke is cluster-wide immediately. docker/sync-inbound-api-keys.sh is deleted; a re-run of this drill needs no file copy before the failover step. Re-measured 2026-09-03 without the workaround: A/B/E/F YES, C/G NO, active-central docker kill takeover 28 s (the earlier run measured 27 s with the file copy), victim rejoined as standby 2 s after restart. Prod wonder-app-vd03 is single-node and unaffected; the QA pair (vq03/vq04) and prod still run Provider=Sqlite pending a deployment record.

Deferred-work register row 107 records §7 item 1 (Children["DelmiaRecv"]) as the prod follow-up.