Files
ScadaBridge/docs/deployment/deployment-records/2026-11-15-gate-52-delmiareceiver-fix.md
T

97 lines
8.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 2026-11-15: #52 `CvdReactor.ProcessRecipeDownload` child-name fix, DEV vd03 + QA vq03/vq04 (Z28061 excluded)
**Owner go:** 2026-11-15, "yeah" to applying #52 (DEV first, then QA). **Z28061 is excluded on both tiers**: another team uses it and no redeploy slot has been agreed.
**Change:** one line in template `CvdReactor` (id 1), script `ProcessRecipeDownload` (id 4): `Children["DelmiaRecv"]` becomes `Children["DelmiaReceiver"]`. The body is `deploy-prep/2026-11-12-gates-52-131/fixed/CvdReactor.ProcessRecipeDownload.cs`. Every other script field is re-sent from `original/CvdReactor.ProcessRecipeDownload.meta.json` and read back unchanged.
**Scope:** management API only, through the ScadaBridge CLI (built from `main`) run from the Mac.
- No binary, no migration, no service restart, no WinRM. Only the redeployed instance actors restarted.
- Not touched: Z28061 (template change only; its deployment was left alone), every other template, script and instance, OtOpcUa, mxaccessgw, HistorianGateway and Overview.
**Tooling change (`apply.sh`, same commit):** every CLI call now passes `--url`, `--username` and `--password` explicitly. Earlier on 2026-11-15, an env-only `SCADABRIDGE_URL` pointing at QA had silently reached **DEV**, because the CLI fell back to its configured default central. Two optional guards were added:
- `SB_EXPECT=<names>` refuses to run unless the listed instances exist on the central, which proves the tier. The full instance list is printed first.
- `SB_SKIP=<names>` never deploys the listed instances. They are left stale.
## Functional check (read-only)
The call is `flow-debug run-script --instance-id 2 --script ProcessRecipeDownload --params '{}'` on **Z28062**, made only after confirming `DelmiaReceiver.ReadyFlag` is **False / Good**. In that state, the script returns before its `try` block, so nothing is written to the PLC.
- The returned text is the same before and after: `Receiver ready flag not set to true`. That is the correct answer when the real flag is False, so the text alone does not prove the fix.
- The evidence is the trace instead. **Before** the fix, the run logged `WARN runtime GetAttribute: attribute 'DelmiaRecv.ReadyFlag' not found on instance 'Z28062'`. This is the #52 defect: a null child, so the hard-coded text came back regardless of the real flag. **After** the fix, the warning is gone, and the script read the existing `DelmiaReceiver.ReadyFlag`.
- Not exercised: the download path with `ReadyFlag` True. It writes `RecipeDownloadFlag` and the recipe fields to the receiver and waits on `RecipeProcessedFlag`. Running it needs a planned recipe download.
- On QA, **Z28063 has `DelmiaReceiver.ReadyFlag` True**. A call there would take the write path, so it was **not** called. Its first real `DelmiaRecipeDownload` will now take the download path, which it did not before.
## DEV vd03 (`http://wonder-app-vd03.zmr.zimmer.com:8085`, DisableLogin)
| Step | Result |
|---|---|
| Tier proof | Instances: SBTEST01, SBTEST02, Z28061, Z28061Sim, Z28062, ZIPSEN. This is DEV's set |
| `check 52` | Live body **ORIGINAL**. Z28061, Z28061Sim and Z28062 not stale (deployed 2026-10-28) |
| Negative control | The trace logs `DelmiaRecv.ReadyFlag' not found` |
| `apply 52` (`SB_SKIP=Z28061`) | Script 4 updated; metadata read back unchanged. Deployed **Z28061Sim** (deployment 97) and **Z28062** (98). Z28061 skipped |
| `check 52` after | Body **FIXED**. Z28061Sim and Z28062 not stale (22:23:20–21Z). **Z28061 `stale=True`**, as intended |
| Trace after | RouteCall and ScriptRun Delivered, with the same ResultText and **no `not found` warning** |
## QA vq03/vq04 (active central **vq04** `:8085`, login `multi-role`)
**The active central is vq04.** `/health/active` returns vq03 503 and vq04 200. On 2026-11-13 it was vq03. Live flow-debug runs are refused on the standby (`STANDBY_NODE`), so every QA call went to vq04. Reads and writes go through the active node either way.
| Step | Result |
|---|---|
| Safety re-check (22:24Z) | The 8 instances to redeploy: `CycleRunning` False/False on both sides, FurnaceTemp 23–25 °C. All SAFE. Open batches: Z28063 L 29049, Z28065 R 29020, Z28068 L 29081 |
| Tier proof | Instances: WarmCanary, Z28061–Z28069, ZIPSEN. This is QA's set |
| `check 52` | Live body **ORIGINAL**. All 9 CvdReactor instances not stale (deployed 2026-11-06) |
| Negative control | Z28062 trace logs `DelmiaRecv.ReadyFlag' not found` |
| `apply 52` (`SB_SKIP=Z28061`) | Script 4 updated; metadata read back unchanged. Deployed **Z28062–Z28069** (deployments 233–240). Z28061 skipped |
| `check 52` after | Body **FIXED**. Z28062–Z28069 not stale (22:24:33–37Z). **Z28061 `stale=True`** |
| Trace after | Z28062: RouteCall and ScriptRun Delivered, with **no `not found` warning** |
| Post-redeploy | All 8 instances 244/244 attributes Good. `CycleRunning` False. The open batches and current steps are unchanged (Z28063 L 29049/step 2, Z28065 R 29020/step 2, Z28068 L 29081/step 6). `ReadyFlag` values unchanged |
## Follow-up
**Z28061 still runs the old script on both tiers.** It shows `stale=True` and still carries the #52 defect. Redeploy it once the other team agrees a slot. The tracking issue is in Gitea (#52 follow-up). The command is either `instance deploy --id 1` (explicit `--url`), or `apply.sh apply 52` without `SB_SKIP`, which is a no-op for the script and deploys only the stale instances.
## Rollback
Per tier: `SCADABRIDGE_URL=<central> SCADABRIDGE_USERNAME=… SCADABRIDGE_PASSWORD=… SB_SKIP=Z28061 bash apply.sh rollback 52`. It refuses unless the live body is FIXED. It writes the original body back, and the instances redeploy onto it.
## Z28061 (2026-11-15, #142)
The owner said: "if its idle you can do rest of #52". So on each tier Z28061 was redeployed **only if it was provably idle at that moment**, which stands in for the slot with its other team. Only the management CLI was used, with an explicit `--url`. Only Z28061 was deployed. `apply.sh apply` was not used, because it deploys every CvdReactor instance.
### DEV vd03: SKIPPED, idle can't be proven
- `apply.sh check 52` at 23:06Z: the body is FIXED, Z28061 (id 1) is `stale=True` (deployed 2026-10-28), and Z28061Sim and Z28062 are not stale.
- `debug snapshot` at 23:06:03Z: 270 attributes, **16 Good**.
- Every idle-check attribute is **Bad, value null**: `CycleRunning`, `MachineCycleID`, `FurnaceTemp` and `MachineBatchID` on both sides, plus `Left/RightDelmiaReceiver.RecipeDownloadFlag`.
- Only `DelmiaReceiver.RecipeDownloadFlag` and `ReadyFlag` are Good, both False, with the value last changed 2026-08-26.
- The rule says Bad quality means not proven idle, so it was **not deployed**. DEV Z28061 still runs the old body (`Children["DelmiaRecv"]`).
- **To unblock DEV:** get Z28061's DEV data source back to Good quality, or have its team confirm it's idle.
### QA vq03/vq04: DONE
- **Active central:** vq03 (`/health/active` vq03 200, vq04 503); the #116 stop-and-hold drill had moved it there. Every call went to vq03.
- **`apply.sh check 52`:** body FIXED, `SB_EXPECT=Z28063,Z28069,WarmCanary` passed. Z28061 (id 1) was the only stale instance; Z28062–Z28069 were already current.
**Idle check** (snapshots at 23:06:18Z and 23:07:25Z, 67 s apart; 244 attributes, all Good):
| Attribute | Left | Right |
|---|---|---|
| `CycleRunning` | False | False |
| `MachineCycleID` | 0 | 0 |
| `FurnaceTemp` | 25 | 25 |
| `MachineBatchID` | 0 | 0 |
- `DelmiaReceiver.RecipeDownloadFlag` and `ReadyFlag`: both False. QA's Z28061 has no Left/Right Delmia receivers.
- Between the two snapshots, the only change was `TrunkPressure` 0.18 → 0.17, which is noise.
**Deploy and verification:**
- **Deploy:** `instance deploy --id 1` gave deployment 241, status 2, done at 23:07:36.81Z in 0.26 s; revision `sha256:2727a432…`.
- **`instance diff`:** `isStale=False`.
- **Snapshot after (23:08:02Z):** 244 attributes, 244 Good, no value change except `TrunkPressure` noise, and no batch, so nothing to preserve.
- **Trace:** `ReadyFlag` re-read as False / Good first, so the script returns before any PLC write. `flow-debug run-script --instance-id 1 --script ProcessRecipeDownload --params '{}'` at 23:08:13Z gave RouteCall and ScriptRun both Delivered, with **no `not found` warning**.
### Result
- **QA:** all ten CvdReactor instances (Z28061–Z28069) run the fixed body.
- **DEV:** Z28062 and Z28061Sim are fixed. **Z28061 is still pending** on #142.